Recent News

Building AirMail Push for Android’s Infrastructure

We’ve been working to develop AirMail Push for Android, our push platform, along with a supporting server-side infrastructure that can handle millions of concurrently connected devices. Like Push Notifications on Apple’s iOS platform, AirMail for Android maintains a persistent socket connection to our cloud, which lets us push out messages to devices in real-time. This brings with it all the complexity of handling hundreds of thousands of persistent socket connections server side, managing the connection on the client side as users pass in and out of coverage or switch between networks, and detecting and closing half-open sockets as quickly as possible.

This article describes the challenges we faced building out the server-side push infrastructure, how they led us to re-design and implement it using a hybrid threaded/evented/queue-based architecture based on Java’s NIO libraries, and a retrospective on this decision now that it’s live.

Lots and Lots of Sockets

In order to push a notification to a device, you need to have a persistent socket connection open to it. Our platform needed to be capable of serving millions of simulatenously-connected devices, which means that our server’s system stack and our software need to elegantly accept, read from, and push notifications out to a very large number of open connections per node. But how many? 10k? 50k? 100k? 500k? 1MM? More importantly, how many nodes do we need to handle this? If one node can only serve 10,000 clients, the venture is cost-prohibitive from the start – just one million devices would require a fleet of 100 servers.

We began our venture into uncharted territory with an implementation in Python based on Eventlet. Eventlet’s a fantastic library that provides support for concurrent task execution through lightweight coroutines. Its socket implementation allows developers to accept and manage an arbitrary number of connections with less difficulty and expensive context switches associated with threaded implementations. With it, the first version of our edge server “Helium” was born.

Eventlet helped us build a Helium whose code was incredibly straightforward, clean, and easy to understand. We love Python and are very grateful for the community of developers who are working to build libraries like this. Our initial implementation of Helium enabled us to accept and send application-level keepalives to roughly 37,000 connected clients before hitting the 1.7GB memory limit of a base EC2 instance. While pleased with this number, the amount of instances needed to meet our capacity requirements would have been prohibitively large and expensive. While we are constantly investing in our infrastructure to build a stable, secure, performant platform, we’re better programmers than we are sysadmins and prefer to write software that scales both efficiently and cost-effectively.

Things That Look Like Sea Monsters

Thus began our search to build a prototype of a more efficient socket server. We considered a variety of options, including a C/C++-based implementation wrapping [e]poll, a threaded or evented implementation in Java or Scala, a spike using Node.js, along with a few others. As expected, the purely-thread-based Java option fell over after a few thousand connections, and the C-based implementation proved too low-level to meet our evolving project requirements.

A quick experiment using Java’s NIO library (“new/non-blocking” IO) immediately turned heads, so we spent several hours fanning out that spike to include three versions:

• A raw Java NIO implementation
• An implementation of the same, atop the Netty library.
• And one more using Netty, but written in Scala rather than Java.

Each of these three spikes vastly surpassed our previous results both in terms of connections achieved, and memory efficiency:

Table 1: Connections Handled Before Failure

Table 2: Memory Efficiency per Connection

Work Smarter, Not Harder

Jumping from 35,000 connections per node on an EC2 Small instance to over half a million on a single EC2 Large marked a huge win in terms of efficiency, infrastructure cost, and administration overhead — the number of connections per node jumped by nearly 1500%. (As a premature epilogue, this number held true after implementing all of our application logic, keepalives, and message passing, which is almost unheard-of in an early test).

As an interesting sidenote, the failure modes we saw in both the Java + Netty and Scala + Netty were unusual; CPU usage spiked to 100% in the Java process, with nothing output to the console. It appeared that an exception was being thrown and silently caught in a tight infinite loop.

Based on these numbers, we pressed on with a Java + Pure NIO implementation. Stay tuned – we’ll have a few posts on our NIO implementation, more info on our methodology and metrics, some things we’ve learned on the way, and all sorts of sharp corners we’ve hit our heads on that you can avoid.

More soon!

Rich experience in infrastructure, social business and open source round out the talented advisory board at Urban Airship. As our company continues to build out our vision for a complete end-to-end mobile services platform, where information and content is readily accessible to anyone with a smart phone or mobile device, Justin Fitzhugh of Mozilla and Dave Hersh of Jive Software bring valuable insight from their current fields. They join an already talent-rich team.

Our mobile services platform is at the heart of the mobile future; we’re building a strong, robust, scalable and secure infrastructure to deliver on the promise of the always-on, real-time mobile experience. As we further refine what we’ve built and continue to execute on our cross-platform strategy, Justin’s experience will be invaluable. He is recognized throughout the industry for his work building a huge internet infrastructure and we’re thrilled to have him on board.

We continue to see a convergence of social + business across industries, and it’s no different in mobile, which blends both the elements of media consumption and interactive, interpersonal engagement. As businesses develop more sophisticated apps that really target their user base, we’re seeing more interest in how driving engagement with app users can deliver solid business value and increase the ROI of a mobile strategy. Certainly, Jive Software has proven the business value of social business and much of the company’s success is owed to Dave’s vision and execution there. So he’s a great champion for us as we continue to provide tools for businesses to get actionable, rich analytical data about the mobile strategies. It doesn’t hurt that Jive is also a Portland-based business!

Justin Fitzhugh is the VP of Engineering Operations for the Mozilla Corporation. He responsible for all Mozilla’s production and corporate infrastructure, including serving the Firefox product to more than 150 million users. In addition to Firefox distribution, his team designs, implements and supports the infrastructure for one of the largest open source organizations in the world. Prior to Mozilla, Justin managed Macromedia’s global datacenter environment. He spends his spare time as an avid pilot, snowboarder and father in the Bay Area.

Dave Hersh is the Chairman of the Board of Directors at Jive Software, responsible for advising the company’s growth strategy and overall direction.

Prior to becoming Chairman, Dave was the Founding CEO of Jive, a Sequoia and Kleiner Perkins backed growth company. In that role, his vision and leadership grew the company from its inception as a small open source project with no revenue to a $55M, pre-IPO company. Jive became the hands-down leader of the Social Business Software revolution, with leadership positions in three separate Gartner Magic Quadrants. Dave has gained a reputation for his deep industry expertise in collaboration, knowledge management and open source.

As mobile developers, one of the most challenging obstacles we face is finding ways to effectively market your applications. Unless your application makes it into the coveted ‘featured’ section of the Android market, you’re on your own.

For this reason Urban Airship now provides developers with a new tool called AirDrop. AirDrop which will aid developers in promoting their applications by making it easy to distribute your applications to your friends, fans, and reviewers.

Here are some ways you might use AirDrop:

• Seed reviewers
• Provide free copies for promotions and or contests
• Share them with people interested in your application
• Send development copies to testers

Once you upload your application to AirDrop, you can create custom redeem codes and links to share and distribute. You’ll be able to see who has redeemed codes and, best of all, we make it extremely easy to get the application onto the user’s device.

If the user clicks on the redeem link from their desktop, AirDrop provides that user with some options on how to navigate to that link from their mobile device. Options for sending the url from their browser to their device include a QR code, Google Talk (aka Jabber), and email. Once the user clicks the redeem code from their mobile device the download starts immediately.

Getting started with AirDrop is easy. There is no sign-up, just login with your Google account, upload your application, and start generating redeem code links for distribution immediately.

Just another tool from Urban Airship to help make your lives easier.

Proper spelling is crucial for the written word, especially when having a party. Probably the word I look up most is Hors D’oeuvres, a standby in the Evite economy. Another social spelling necessity is RSVP (all caps? periods after the letters? And what does it mean, anyway?) Luckily, when I’m texting out my invites, I have the dictionary.com iPhone app as a handy reference to these and other social dilemmas.

No, one simply could not host a proper party without the dictionary.com iPhone app.

There’s a Word for That

The iPhone app’s cool Word of the Day Feature also helps out in the chitchat hour of the party, when engaging in small talk. Say for instance, someone is describing her latest culinary triumph, a spatchcock chicken. You probably wouldn’t the slightest clue what she means unless you chose to receive the optional Word of the Day Push Notification from dictionary.com, in which case you’d know right away that “spatchcock” means “fowl that has been dressed and split open for grilling.” Then you could respond with a witty retort.

I venerate the Word of the Day, which, it so happens, is delivered via Urban Airship’s AirMail Push service. We are thrilled to be working with Dictionary.com to enrich the vocabularies of the millions of app users on the Apple iOS and Android platforms by delivering a fresh cool word every day. The free Dictionary.com app delivers world-class reference content from Dictionary.com and Thesaurus.com including nearly 1,000,000 words and definitions and 90,000 synonyms and antonyms.

Dictionary.com push notifications don’t only help users learn new words. They also help dictionary.com with its mobile business strategy, enabling the company to interact with users in real-time, directly to their pockets. And in a way that is non-meretricious because it is opt-in.

Just like a gifted party host, the app is well mannered.

There’s lots of chatter today about the issues Time Inc is having with Apple and the subscriptions portion of its iPad apps. As Peter Kafka writes today on All Things Digital, Apple rejected the subscription version of the Time Magazine app, surprising Time executives and forcing the publisher to only sell single copies of the magazines, meaning users need to download and pay for each issue as individual transactions. All purchases use iTunes as a middleman.

Considering this is an entirely new medium for content, it’s no surprise that the industry is working out some kinks. Given the debate we wanted to share some of our insight, developed through engagement with our customers in the publishing industry, who are integrating our solution as we speak for their upcoming iPad magazine apps, which plan to offer subscriptions.

In a nutshell, when it comes to taking publications to iPads and other mobile devices, publishers face several challenges:

• No way to offer subscriptions (only single-issue downloads)
• No clear way to track conversion rates when subscriptions end
• No way to offer magazine subscriptions as gifts
• No easy way to alert app users to new content

Publishers want subscriptions so they can have valuable customer data – this is how they demonstrate value to advertisers and has been thus for eons. Users want subscriptions because it costs a lot less per issue. I know I much prefer the $1 / issue of Vanity Fair each month than the $5 or whatever at the newsstand.

How Urban Airship is addressing these challenges through StoreFront

Our subscription feature, a component of its in-app purchase offering, is designed to manage all aspects of the subscription flow to mobile devices, including delivering new content, alerting subscribers to new content and tracking the billing aspects associated with ongoing downloads. So the team here built a great way to manage the flow of content. With StoreFront, they’ve also created a way to let customers avoid two main issues Apple implies are the reason for the rejection.

1) Concern about customer data. We don’t keep track of any data on the customers. Meaning we don’t actually store customer email addresses or UDIDs, just hashed versions. Publishers might obtain information about their target audience through their own devices, but StoreFront doesn’t help at all with that. We simply manage who has access to what content based on the hashed data and then deliver the content.

2) Revenue share. StoreFront subscriptions still funnel through iTunes; users get new content from within the app. The billing set up is the same as with any app delivering content via in-app purchase.

So that’s how we are helping publishers while staying in compliance with all of Apple’s published guidelines. We can’t promise perfection, but we know we’re close. We hope to see the publication apps from our customers available soon. (personally, I can’t wait; these are apps of beauty). When the apps are available, we’ll be sure to mention it. In the meantime, we are watching how this plays out very closely.

Over Five Hundred Million Served

We’re excited to announce that we just surpassed the 500 million mark for the number of notifications we’ve sent on behalf of our clients’ mobile apps. It seems like only yesterday that Silicon Florist broke the story about our 100 millionth message, and here we are, only a few rainy months later, with another important milestone to share. With all of the other new product and customer announcements that have been keeping us busy, this seems like a great time to catch a breath and reflect on the past year.

It’s not us, it’s you

As much as we like to toot our horn here at UA, the real success story here is for our customers, so we should be saying congrats to you! If I had a nickel for every UA customer that came up with an amazingly creative new way to use Urban Airship Push, I would have a ton of nickels.

Lesson learned

Push Notifications are no longer an afterthought or just a bit of add-on technology for app developers.  In the early days, many of our customers were adding Push in their iPhone 2.0 or 3.0 releases.  Today, push notifications are central to the creative process, being a critical component of app design, user experience, and brand messaging.

Not only that, but iPhone and other smartphone users not only recognize what Push is, but also have come to expect is as part of the app experience. We are actively engaging with our BlackBerry and Android customers to bring the same push experience to mobile users across multiple platforms.

A look at the numbers

A few highlights from the past year:

• June 17th, 2009. iPhone OS 3.0 released, introducing Apple’s Push Notification Service (APNs)
• June 17th, 2009. Tap Tap Revenge, utilizing Urban Airship, becomes the first app in the App Store to use Apple’s new service.
• July 30th, 2009. Urban Airship hits 1 million messages
• Feb 5th, 2010. Urban Airship hits 100 million messages
• April 8th, 2010. Apple announces that 10 Billion messages have been sent through APNs in 9 months (Urban Airship powered over 2% of them!)
• June 17th, 2010, Urban Airship sends message number 500 million

Looking forward

As we continue to grow our customer base, make it stupid-simple for developers to use our stuff, and provide the world’s most reliable and scalable 3rd-party push notification integration service, we’re pretty certain that we’ll be back here talking about a billion messages very soon. In the meantime, we’re working really, really hard on some other stuff that will knock your socks off.

Next up: Expanding our Blackberry Push footprint, powering Push for the Android Market, rich-media mobile messaging with AirMail, and much, much more.

Up, up, and away! Thanks for your support!

Since we announced the developer preview in March, we’ve seen many fantastic beta implementations for AirMail. Today we take the wraps off AirMail and announce its public release.

With the release of AirMail, we’re able to show off our launch partners, the Democratic National Committee and LivingSocial. These apps will use AirMail’s rich interactive and customizable push notifications to bring something innovative to the app world.

The DNC has integrated AirMail into both its Democratic Party and Organizing for America iPhone/iPad apps that were released earlier today. AirMail gives them a great way to engage and mobilize (pun intended) their constituency like never before. Our AirMail service powers the apps’ Alerts, which will notify you of breaking news, events in your local area, and ways you can get involved. Like all AirMail messages, these Alerts allow you to take some sort of action, view video, or RSVP for events — all from within the messages themselves.

Organizing for America has an ambitious plan to reach out to voters this election season and—through the Vote 2010 campaign—reconnect with the millions of people who were inspired to participate in the political process for the first time in 2008. This is smart thinking; so many people get the majority of their news and information from mobile apps that an interactive and customized app is the perfect way to connect with voters on-the-go.

LivingSocial has integrated AirMail into its iPhone app to help power their coupon offers and give them a new way to deliver offers right to users’ pockets. LivingSocial is leading the industry in socializing the coupon, encouraging users to share their deals with friends and contacts – AirMail makes it even easier for LivingSocial users to get great deals in their markets. The LivingSocial app is live in the iTunes app store; you can find AirMail under the “Inbox” tab.

Push notifications are great – we love them of course. They solve a key challenge for mobile publishers who want to engage their mobile app users after they have installed the app. In talking with our customers, they wanted more function from push. They wanted a persistent inbox for the notifications their apps send to users. They wanted to send richer notifications, with things like HTML, video and audio. They wanted more advanced messaging features, such as return-receipt or secure messaging. They wanted a way to directly engage in two-way interactions to obtain real-time feedback, from inside the app.

So we created AirMail. It gives our mobile publishing friends all of that and more.

And we’re just warming up.

If you’d like to learn more about AirMail, please visit our product page or get in touch with someone on our team who can help you tailor a solution for your mobile app today.

PhoneGap is an awesome open-source project by the guys at Nitobi that provides tools for building cross-platform mobile applications using HTML and Javascript. Using the plugin provided here you can begin sending push notifications from Urban Airship to your PhoneGap application quickly and painlessly.

The first thing you’ll need to do is download: the plugin and sample code.

To get started, create a folder named Plugins in your project folder if it does not exist. Then add PushNotification.h and PushNotification.m to it. Now copy PushNotification.js to /PROJECT_PATH/www/. Note that the plugin sample code also contains an index.html. We won’t be using it in this example, but is provided for reference.

Import PushNotification.js via a script tag in the <head> of your application.

<script src="PushNotification.js" type="text/javascript"><!--mce:0--></script>

Now in some <script> tags you’ll want to add the following code to your application. The block below handles registering your application for push notifications with Apple and then sending the device token Apple returns to Urban Airship for registration.

// app key and secret for your Urban Airship application
var HOST = 'https://go.urbanairship.com/';
var KEY = '';
var SECRET = '';
 
// When this function is called, PhoneGap has been initialized and is ready to roll
function onDeviceReady() {
    navigator.pushNotification.startNotify();
    registerAPN();
}
 
// Register device for push with Apple.  success and error callbacks passed along
// with the push features your app is requesting
function registerAPN() {
    navigator.pushNotification.register(successCallback, errorCallback,
        { alert:true, badge:true, sound:true });
}
 
// Callback when receiving notification.  This method is only called
// when your application receives a push notification while it is open.
PushNotification.prototype.notificationCallback = function (notification) {
	var msg = '';
	for (var property in notification) {
		msg += property + ' : ' + notification[property] + '\n';
	}
	alert(msg);
};
 
// when register APN succeeded
function successCallback(e) {
    alert("Device registered. Device token: " + e.deviceToken);
    registerUAPush(e.deviceToken); //register token with UA
}
 
// when register APN failed
function errorCallback(e) {
    alert('Error during registration: '+e.error);
}
 
// register with Urban Airship push service
function registerUAPush(deviceToken) {
    // open the client and encode our URL
    var request = new XMLHttpRequest();
    request.open('PUT', HOST+'api/device_tokens/'+deviceToken, true, KEY, SECRET);
 
    // callback when request finished
    request.onload = function() {
        if(this.status == 200 || this.status == 201) {
            alert('UA push service successfully registered.');
        } else {
            alert('Error when registering UA push service.\nError: '+this.statusText);
        }
    };
    request.send();
}

You can also check out index.html included in the sample code for a more detailed implementation.

Now, you’re good to go. Your application has registered that device with Urban Airship. All you have to do to send a push notification to that device is send an HTTP POST request to Urban Airship’s REST API.

Once you have this working, it’s trivial to take advantage of other Urban Airship Push features such as tags, aliases, broadcast, and auto-badge.

We’re looking forward to seeing what kind of great things people build using PhoneGap with Urban Airship – be sure to drop us a line when your app goes live!

Appcelerator is a popular method of quickly building iPhone applications using HTML and javascript. We’ve worked with the Appcelerator folks to bring you some basic steps for how to integrate Urban Airship into your application.

The first thing to do is make a call to registerForPushNotifications which will asynchronously return the result of the registration to one of the callback functions you provide.

var APP_KEY = '';
var APP_SECRET = '';
 
Titanium.Network.registerForPushNotifications({
    types:[
        Titanium.Network.NOTIFICATION_TYPE_BADGE,
        Titanium.Network.NOTIFICATION_TYPE_ALERT,
        Titanium.Network.NOTIFICATION_TYPE_SOUND
    ],
    success: successCallback,
    error: errorCallback,
    callback: messageCallback
});

Read more »

Urban Airship Accepting Beta Users for Android Push Notification Service

Communicating with your users via their mobile devices give you an extremely powerful channel to user attention. Urban Airship’s technology allows you to engage with users—via their mobile apps—cheaply, easily, and effectively. We started by offering Push for apps on the iPhone, then we shipped BlackBerry support. Today, we’re happy to announce that, furthering our goal of “one API, any device,” we’re accepting beta applicants for our Android Push offering.

This means that with one simple API, you’ll have the power to communicate to your users across three of the most popular mobile operating systems. Your multi-platform strategy just got a lot easier to implement.

Even if you don’t have any interest on being on other platforms, our API brings a lot of value. Aside from the ease of integration and our amazing support, our solution gives you tagging and broadcast capabilities, the ability to automatically deliver messages based on feeds, and other advanced features designed to make your job easier.

We’re also bringing our push notification technology to all versions of the Android operating system, so your users don’t have to have the latest updates or devices in order to take advantage of these new features. Not only that, our notification solution won’t require having the Android Marketplace installed or having a Google Account.

As always, we’re here to make your lives easier — let us worry about scaling, monitoring, maintenance, and all the other hassles that you have to deal with if you implement your own messaging solution.

If you want to be first in line when we open up our beta, sign up below and we’ll be in touch.