Archive for the ‘OAuth’ Category
Using REST APIs from R
It’s hard to read a website, blog post or even mainstream business press article without coming across the term ‘big data’. Big Data is one of those terms that means nothing and everything all at once, and for that reason alone, you should pay attention to it. When it comes to the how of big data, it’s equally hard to avoid bumping into R, the open source statistics and computational environment. If you’re going to describe and model the behavior of your customers in a big data initiative, R is one tool that you need in your software toolbox.
Data is everywhere, and increasing, data of all kinds on customer engagement is available through REST APIs. I started a little project in my spare time to bring data available via REST interfaces into R, to set the stage for doing what I expect to be some fairly sophisticated model building. The rest of this post is a quick introduction to how to work with REST APIs in R.
At its most basic, calling a REST API to obtain data involves making an HTTP GET request to a server. If the call succeeds, you’ll have a document that contains the requested data. In R, the best way to make these requests is by using RCurl. The RCurl package is – you guessed it – an R interface to curl. Once you’ve installed it into your R environment, getting data from REST APIs is pretty straightforward.
For my project, I started with the Constant Contact API, partly because I work for Constant Contact on the Web Services team, and party because the API makes available exactly the kind of data that you typically want to analyse in a marketing big data project – specifically, sends, clicks, opens and the like for marketing campaigns. The current v1 API returns XML, so I also installed the XML package into my R environment (though I haven’t done much with it yet). To install the packages use the following command in R:
> install.packages(‘RCurl’, ‘XML’)
To load these packages, use:
> library(‘RCurl’)
> library(‘XML’)
Once these preliminary tasks are taken care of, there are just 2 steps required to get campaign data from the API and into R:
1. Obtain an access token. Constant Contact uses OAuth 2.0 for authentication, as do many other public REST APIs. There’s no good way to get a token from inside R, so I used the client flow with a little bit of javascript to get the token in my browser, and then just saved it for use in R. See here for details on how to get access tokens. If you’re building an app that analyses data from multiple Constant Contact accounts, you’ll need the owners of those accounts grant access to your app in order for you to obtain access tokens. But for now, sign up for a trial and use your own account.
2. Call an HTTP endpoint using RCurl. This is very easy. For my initial test, I wanted to get the list of available email campaigns, so that I could later iterate over the list and get the campaign statistics for analysis. The call is:
campaignsXML = getURL(“https://api.constantcontact.com/ws/customers/{username}/campaigns?access_token={token}”)
{username} : replace with the name of the account for which you have the access token
{token} : replace with the actual access token granted to you by the account owner
This issues the HTTP GET request, and puts the resulting XML response into the R vector ‘campaignsXML’, ready to be processed further. That’s all there is to it.
In my next post on this topic, I’ll show you how to parse the XML to get it into a more usable form using the R XML package.
Happy Model Building!
– Matt
Building an OAuth enabled website using Java
For me, it’s always been about the APIs. I write them. I use them. I have built and continue to grow my career around them. So as it became clear over the past few years that OAuth has become the de facto standard for API authentication, I took the plunge and figured out how to do practical things with OAuth-based APIs. Since Java is my preferred language, and since I’m now the Product Manager at Constant Contact for the Platform and Partner Integrations, the natural path for my latest project was to build a website using Java that lets users authenticate to Constant Contact via OAuth.
That said, here’s the basics of the application and how it works. The technology stack looks like this:
Web UI: HTML, CSS, JavaScript
Application Services: Java Servlets, JavaScript (Ajax), Scribe-Java (OAuth Java library), REST calls to the Constant Contact API
Database (for storing OAuth tokens and secrets): Hibernate, MS-SQL Server
These are all tools that should be in any programmers toolbox, so the application also serves as a good training app and sample code if you’re trying to get familiar with this stack. The code is available on Github (you do have a Github account, don’t you??). Just fork my repo and you should be ready to go.
Since the main point of this article building an OAuth based website, let’s get right into it. If you want your users to display their data on your website, you as the website owner must let your users grant access to their data. In my case, the data is coming from Constant Contact, so I implemented what is known as the ‘web flow’ for authentication. The flow uses a series of callbacks and goes roughly like this:
– Get a request token from Constant Contact
– Redirect the user to Constant Contact’s OAuth endpoint and ask the end user to authenticate and grant access to their data
– Exchange the authenticated request token for a valid access token and secret and have OAuth redirect the user back to your site.
Now, most of us are not hard-core security programmers, and trust me, if you (like me) prefer to focus on the functionality of your app and not on plumbing, you should use an OAuth library. In my case I chose scribe-java, an excellent open source library for Java OAuth programmers. I submitted the Constant Contact API to the project so that the rest of you can easily authenticate to Constant Contact. You can get scribe-java from github at this url.
I handle the callbacks through two Java servlets. AuthServlet.java makes the initial request for an OAuth Request token, and if successful, redirects to Constant Contact to let the user authenticate and grant access. The basic code looks like this:
OAuthService service = new ServiceBuilder()
.provider(ConstantContactApi.class)
.callback("http://localhost:8080/CTCTWeb/OAuthCallbackServlet.do")
.apiKey(apiKeyProperties.getProperty("apiKey"))
.apiSecret(apiKeyProperties.getProperty("apiSecret"))
.build();
httpsession.setAttribute("oauth.service", service);
Token requestToken = service.getRequestToken();
httpsession.setAttribute(“oauth.request_token”, requestToken);
String confirmAccessURL = service.getAuthorizationUrl(requestToken);
System.out.println(confirmAccessURL);
try {
res.sendRedirect(res.encodeRedirectURL(confirmAccessURL));
} catch (Exception e) {
System.out.println(e.getMessage());
}
The simplicity of the code is due to scribe-java. A couple of things to notice. First, we tell scribe-java what the callback URL is when we create the service object. Second, we store the request_token in the httpsession object. This isn’t strictly necessary (we could just store the request token secret, and then reconstruct the request token in the callback), but it is convenient. The reason for this has to do with the scribe-java Token class – it isn’t really a Token per-se, it’s a token-secret pair. I suspect that the author of the library did it this way because token-secret pairs are ubiquitous in OAuth. In any case, when the callback is invoked, it winds up in OAuthCallbackServlet.java. The relevant code looks like this:
String oauth_token = req.getParameter("oauth_token");
String oauth_verifier = req.getParameter("oauth_verifier");
String username = req.getParameter("username");
if (oauth_verifier.length() > 0) {
Verifier verifier = new Verifier(oauth_verifier);
HttpSession httpsession = req.getSession(true);
OAuthService service = (OAuthService) httpsession.getAttribute(“oauth.service”);
Token requestToken = (Token) httpsession.getAttribute(“oauth.request_token”);
httpsession.setAttribute(“username”, username);
Token accessToken = service.getAccessToken(requestToken, verifier);
httpsession.setAttribute(“oauth.access_token”, accessToken);
Long accessTokenId = null;
AccessToken at = new AccessToken();
at.setLoginName(username);
at.setAccessToken(accessToken.getToken());
at.setSecret(accessToken.getSecret());
Date dt = new Date();
Timestamp ts = new Timestamp(dt.getTime());
at.setModifiedDate(ts);
Session session = HibernateUtil.getSessionFactory().openSession();
Transaction transaction = null;
try {
transaction = session.beginTransaction();
accessTokenId = (Long) session.save(at);
transaction.commit();
} catch (HibernateException e) {
transaction.rollback();
e.printStackTrace();
} finally {
session.close();
}
}
Again, refer to the full source code on github. There’s several things going on here. First, the entry point to the call back is a servlet URL, which the Constant Contact OAuth implementation has called with three parameters. The oauth_token is the same request token that you generated earlier (but just the token, not the secret). Because I stored the request-token/secret pair earlier, I just ignore this parameter (but see the earlier comment – you could store the secret and then reconstruct the scribe-java Token object). The verifier is a string that the OAuth server creates when the user authenticates and then grants access. The magic is in the call to service.getAccessToken, where you trade in the (now verified) request token for a valid access token. Once you have the token, you need to do something with it so that the next time your user visits the website they can access their Constant Contact data. In my case, I opted to store the username, access token and the token secret in a SQL database. The schema for the database is pretty simple, and Hibernate makes it fairly trivial to store and retrieve the token and secret.
That’s about it for now. Happy coding!
– Matt
Technistas 