Cloud computing seems to be one of the big industry buzzwords.  But what’s all the buzz about?

Well, for one thing, $50/application and a potential $15K in prize money, thanks to Elance and Microsoft… but more on that later.  In addition to crafty gimmicks in which yours truly hopes to take part, cloud computing seems to be catching on in organizations and among professionals seeking to reduce time to market, equipment costs and service outages.  Several major players are on board, including Microsoft (Windows Azure), Amazon.com (Amazon Web Services), Google (Google Apps, AppEngine) and Salesforce.com (Force.com).  There are many other players and I’m not intentionally leaving any off the list – for the detail-oriented (and marketing managers), feel free to add a comment promoting your service of choice.  I don’t mind… really.

I’ve personally dabbled with Windows Azure and Amazon Web Services.  It’s worth discussing both, as they represent fairly different service models.  Amazon Web Services encompasses a number of offerings within the AWS brand (like Mechanical Turk, my personal favorite), but is most well-known for EC2, the elastic compute cloud.  To put it as simply as possible, imagine an unlimited number of servers that you had access to whenever you needed to scale up or down your applications and services.  Essentially, that’s what Amazon offers with EC2.  It’s akin to an infinite supply of ready-to-use servers in a remote data center – you can access the machines remotely, set up software and services and be up and running.  This is often referred to as “infrastructure as a service.”  On the other end of the spectrum, you have Windows Azure, often described as “platform as a service.”  To be frank, I hear that most often from people who are not fond of Microsoft, so I’m not sure how far removed from Amazon’s service Azure truly is.  Many of the services available to users are mirrored; if Amazon has a data storage service, Microsoft’s is comparable, both in terms of functionality and price.  However, I have heard that the encapsulation of some functionality – you don’t create “from scratch” images in Windows Azure – makes it easier to get up and running, while limiting your overall flexibility.

For the past few months, I have been attending the recently organized Amazon Web Services Chicago Meet-up to learn more about AWS and cloud computing.  For those of you in the area, I encourage dropping in for the January meeting.  Everyone is very friendly, the breakfast hits the spot and lots of knowledge is served up.  On the other hand, if you are into Windows Azure, you might check out this promotion that Microsoft and Elance are offering (which I eluded to above… you were patient enough to read everything first, right?).  Yesterday night, I got an email from Elance inviting me to participate in a special project to create a Windows Azure application.  Every successful proposal would receive $50 on completion of a live project – it’s brilliant and everyone wins.  Microsoft has several new developers using and promoting Azure, Elance has a bunch of new professionals seeking projects (not to mention the publicity from this event) and I get $50 – sweet!

Stay tuned – once I get approved, I will share details of my Windows Azure application, which everyone will be welcome to beta test.

In case you missed it, the latest AppEngine for Java release adds support for JAXB (the single most requested feature according to Google’s announcement!). In addition, AppEngine 1.2.8 significantly beefs up ORM support, now supporting inheritance, IN queries, and much more.

I had already found workarounds to these limitations for my current project, but the latest release further demonstrates Google’s commitment to a superior platform and eliminates many more excuses for not giving it a try.

Much of Java’s appeal (not to mention JSF, ORM, and GWT with a little help) is the ability to code in terms of rich domain objects such as the enumerated type. My next several blog posts will demonstrate the use of enums throughout a GAE+GWT application, beginning with persistence.

According to the AppEngine docs,  Java enums are not one of the core value types, which might lead one to believe that they’re not supported. However, closer examination reveals that enums are indeed supported by DataNucleus JDO and JPA, which take care of mapping enums to Datastore core value types String (JDO + JPA) or Integer (JPA).

JDO

JDO mapping requires no special annotations and persists the enum as a String by name. One of the DataNucleus doc pages (search for “ordinal”) suggests it may be possible to persist by ordinal instead; however, I haven’t found this documented for JDO. Persistence by name is likely better, anyway, as it allows you the flexibility to add new enum values later in any ordinal position.

	public enum ScheduleMethod {RANDOM, FILL, CUSTOM};
	@Persistent
	private ScheduleMethod scheduleMethod = ScheduleMethod.RANDOM;

Note that Google does have a unit test for enum persistence in JDO.

JPA

JPA mapping requires the use of the @Enumerated annotation, which additionally lets you specify whether to map by name (String) or ordinal (Integer).

Roll-your-own

Of course, you can always roll your own mapping strategy if for some reason you need special behavior. With AppEngine Datastore, you can persist any object class by making it Serializable or Embedded, although the former has the disadvantage that you can't use object properties in queries. You can also create a custom enum class and persist the enum by any core value type as in the example below.

	public static enum Frequency
	{
		DAILY(1),
		WEEKLY(7),
		MONTHLY(30),
		QUARTERLY(90),
		YEARLY(365);

		private int periodDays;

		Frequency(int periodDays)
		{
			this.periodDays = periodDays;
		}

		public int getPeriodDays()
		{
			return periodDays;
		}

		public static Frequency getValue(int i)
		{
			for (Frequency f : Frequency.values())
			{
				if (f.getPeriodDays() == i)
					return f;
			}
			throw new IllegalArgumentException("Invalid frequency period days: " + i);
		}
	}

	@Persistent
	private Integer periodDays = Frequency.WEEKLY.getPeriodDays();

	/**
	 * Convenience method to get frequency as an enum
	 * @return Frequency
	 */
	public Frequency getFrequency()
	{
		return Frequency.getValue(this.getPeriodDays());
	}

	public void setFrequency(Frequency f)
	{
		this.setPeriodDays(f.getPeriodDays());
	}

It appears that some of the startup delay I had previously blamed on GWT may actually have been due to the phenomenon that is popping up on the AppEngine forums lately. In order to conserve resources, AppEngine doesn’t keep your JVM “hot” unless it’s getting a lot of traffic, and it takes a few seconds to swap it back in. The resulting cold start delay, it would appear, is part of the price of “free.”

See this thread for more info.

I continue to be blown away by the power and EASE of Google AppEngine: one-button deploy from Eclipse, all the admin tools built right in (log viewer, data viewer, task queues, cron jobs, etc.), a graphical dashboard for monitoring performance and usage quotas, and even a system status page that lets me know how the Google infrastructure is performing.

I’m convinced this is a killer platform on which to deploy applications, especially for small businesses already moving into the cloud with Google Apps. And wouldn’t you know, you can already link AppEngine and Google Apps in two important ways:

1. Your AppEngine apps can show up on your Google Apps home page. This happened for me automatically when linking my AppEngine app to a custom domain name using my Google Apps account.
2. Your AppEngine apps can authenticate Google Apps users.

The only problem is, many of the small businesses likely to use Google Apps are doing so precisely because they don’t have an IT staff to maintain servers, email, etc. So they’re certainly not going to have software developers on staff to write custom AppEngine applications for their businesses. But there are plenty of folks like myself willing to write such apps, and AppEngine is already well suited for delivery of pre-packaged applications. It’s already got the control panel and admin tools built in, doesn’t require any database set up, and it’s easy to upload code.

All that’s needed is a marketplace for these applications. That’s where an AppStore comes in. Imagine if you could just sign up your business for a Google Apps account and select from thousands of pre-built applications for managing your business with the same ease as downloading an app for your iPhone. As a developer, I’d love to market such applications, and what better place to reach them than their Google Apps control panel? I’d be ecstatic to give Google a cut of the action! Besides their cut of the application purchase, Google gets more Google Apps sign-ups and sells more AppEngine resources. Google is already a market maker for small business applications through the core search engine and accompanying ad revenue, but a Google AppStore would be far more efficient for both buyer and seller. An AppStore would make many niche applications economically viable by eliminating fixed costs up front and enabling super-efficient marketing, thus realizing the much-written-about “long tail” of software development.

Google, are you listening? You’ve got something here! AppEngine is fantastic, and an AppStore would take it through the stratosphere!

I’ve been able to get Deferred.defer() working by Base64 encoding the serialized object stream for the task before queuing it. I think something in the task queue processing chain doesn’t handle raw binary data. I’ll post a link to the code once smarter folks than I have had a chance to discuss this and review it.

You can follow the discussion on the AppEngine forum.

As I hoped (sort of), the weird serialization error I wrote about a couple days ago came back, and I don’t think it’s related to a field name. Might do some more testing after turkey tomorrow…

My previous post discussed a very cool way to do background processing in AppEngine using Deferred.defer. Unfortunately, the AdminEmailTask example I posted results in a cryptic error when the task is dequeued and the Deferred servlet attemps to deserialize it:

com.newatlanta.appengine.taskqueue.Deferred deserialize: invalid type code: 00

After nearly a day’s worth of experimentation, I am really weirded out by what I’ve found: the problem seems to be the name of the private field “msgSubject” in AdminEmailTask. I tried shorter names, longer names, and other names without any problem. I changed the class name, moved it into a different package, and generally pulled my hair out until I simply tried changing the name of the field. At first, I thought it was this JDK bug, but I wasn’t using an array type. Nevertheless, I subclassed ObjectInputStream and overrode the resolveClass method as per the bug’s workaround, only to find out that AppEngine’s implementation of ObjectInputStream doesn’t call the overridden resolveClass and throws a security exception on enableResolveObject(true).

I can only guess that the field name “msgSubject” results in a binary sequence that is somehow special to AppEngine’s implementation of ObjectInputStream, or perhaps it’s a bug in the JDK.

At any rate, if you get the java.io.StreamCorruptedException with invalid type code: 00, try renaming your class fields! I really, really hope I’m mistaken about this.

How do you do background processing in an AppEngine environment? The AppEngine experimental Task Queues API provides a way using Web hooks, but it’s a bit of a pain because you have to package each background task as a servlet.

Comes Vince Bonfanti of New Atlanta to the rescue with a Java implementation of Nick Johnson’s original Deferred.defer implementation in Python. Now queuing a task for background processing is as simple as creating a task and calling Deferred.defer(task). Here’s an example task to send an email to the application administrator:

package com.roa.server.service.task;

import java.io.IOException;
import java.io.UnsupportedEncodingException;
import java.util.Properties;

import javax.mail.Message;
import javax.mail.MessagingException;
import javax.mail.Session;
import javax.mail.Transport;
import javax.mail.internet.AddressException;
import javax.mail.internet.InternetAddress;
import javax.mail.internet.MimeMessage;
import javax.servlet.ServletException;

import com.newatlanta.appengine.taskqueue.Deferred;

	public class AdminEmailTask implements Deferred.Deferrable
	{
		private String msgBody;
		private String msgSubject;

		public AdminEmailTask(String msgSubject, String msgBody)
		{
			this.msgSubject = msgSubject;
			this.msgBody = msgBody;
		}

		@Override
		public void doTask() throws ServletException, IOException
		{
			Properties props = new Properties();
			Session session = Session.getDefaultInstance(props, null);

			try
			{
				Message msg = new MimeMessage(session);
				msg.setFrom(new InternetAddress("admin@example.com", "admin"));
				msg.addRecipient(Message.RecipientType.TO, new InternetAddress(
					"admin@example.com", "admin"));
				msg.setSubject(msgSubject);
				msg.setText(msgBody);
				Transport.send(msg);
			}
			catch (AddressException e)
			{
				throw new ServletException(e);
			}
			catch (MessagingException e)
			{
				throw new ServletException(e);
			}
			catch (UnsupportedEncodingException e)
			{
				throw new ServletException(e);
			}

		}
	}

And here’s an example of queuing the task inside a gwt-dispatch ActionHandler:

package com.roa.server.handler.admin;

import java.io.IOException;

import net.customware.gwt.dispatch.server.ActionHandler;
import net.customware.gwt.dispatch.server.ExecutionContext;
import net.customware.gwt.dispatch.shared.ActionException;

import com.google.inject.Inject;
import com.newatlanta.appengine.taskqueue.Deferred;
import com.roa.admin.shared.rpc.AdminEmailAction;
import com.roa.admin.shared.rpc.AdminEmailResult;
import com.roa.server.service.task.AdminEmailTask;
import com.turbomanage.gwt.server.PMF;

public class AdminEmailHandler implements ActionHandler<AdminEmailAction, AdminEmailResult>
{

	@Inject
	private PMF pmf;

	@Override
	public AdminEmailResult execute(AdminEmailAction action, ExecutionContext ctx)
		throws ActionException
	{
		AdminEmailTask adminEmailTask = new AdminEmailTask(action.getEmailSubject(), action
			.getEmailBody());
		try
		{
			Deferred.defer(adminEmailTask);
		}
		catch (IOException e)
		{
			throw new ActionException(e);
		}
		return new AdminEmailResult();
	}

	@Override
	public Class<AdminEmailAction> getActionType()
	{
		return AdminEmailAction.class;
	}

	@Override
	public void rollback(AdminEmailAction arg0, AdminEmailResult arg1, ExecutionContext arg2)
		throws ActionException
	{
		// TODO Auto-generated method stub

	}

}

You might ask why you’d use a deferred task like this to send an email? In this example, there really is no reason to, but imagine you were sending 1000 emails at once. In order to guarantee you won’t exceed AppEngine’s free quota of 8 recipients / minute, you could set up your deferred queue in queue.xml with that rate and then call Deferred.defer() for each recipient rather than calling the mail API directly 1000 times in a loop. That way, the AppEngine task queuing facility throttles your mail API calls.

Here’s a link to the whole discussion thread and Vince’s code.

If you’d like to see this added to the Task Queues API, please star this issue.

Someone at last night’s AJUG meeting articulately expressed what I’ve been thinking for the last several weeks as I’ve been learning Google AppEngine: this platform gives the independent software vendor (aka “the computer guy at your church”) a new lease on life. Why will AppEngine bring back affordable custom programming for small business?

1. There are no servers to configure. Sys admin and tech support are the bane of every developer’s existence, which makes AppEngine a developer’s dream. Code, test, and deploy. No sys admin needed.
2. There’s nothing to buy up front: no servers, software, firewalls, backup solution–not even hosting fees until you exceed AppEngine’s generous usage limits.
3. You can log on to your AppEngine application with your Google account (Gmail) or Google Apps login. For many businesses, Google Apps is a compelling alternative to running your own file and document servers and paying for MS Office on every desk. If everyone in your business already has a Google Apps account, it’s a no-brainer to use AppEngine for your custom apps, too.

As more folks learn how to create AppEngine apps in Python or Java, I expect there to be low-cost applications you can purchase for your business as well as pre-packaged applications that developers can buy and customize for your needs. The fabled long tail of software niche marketing is set to EXPLODE.

Shameless plug: if you’ve always dreamed of having software that could do (whatever) for your business, but thought it was probably too expensive, drop me a note. The game has changed.

Google AppEngine’s datastore supports a variety of simple Types and Property Classes by default. However if we want to store something like a dictionary, we typically have to serialize it and store it as a Blob. On fetching we will de-serialize it. While this approach works, it is repetitive and somewhat error-prone.

Wouldn’t it be great if there is a SerializableProperty class that can handle this automatically for us? It doesn’t exist but according to this article, it is easy to create our own customized Property classes. So here is a simple implementation of SerializableProperty that worked for me:

import cPickle as pickle
import zlib
from google.appengine.ext import db

class SerializableProperty(db.Property):
  """
  A SerializableProperty will be pickled and compressed before it is
  saved as a Blob in the datastore. On fetch, it would be decompressed
  and unpickled. It allows us to save serializable objects (e.g. dicts)
  in the datastore.

  The sequence of transformations applied can be customized by calling
  the set_transforms() method.
  """

  data_type = db.Blob
  _tfm = [lambda x: pickle.dumps(x,2), zlib.compress]
  _itfm = [zlib.decompress, pickle.loads]

  def set_transforms(self, tfm, itfm):
    self._tfm = tfm
    self._itfm = itfm

  def get_value_for_datastore(self, model_instance):
    value = super(SerializableProperty,
        self).get_value_for_datastore(model_instance)
    if value is not None:
      value = self.data_type(reduce(lambda x,f: f(x), self._tfm, value))
    return value

  def make_value_from_datastore(self, value):
    if value is not None:
      value = reduce(lambda x,f: f(x), self._itfm, value)
    return value

Usage is as simple as this:

class MyModel(db.Model):
  data = SerializableProperty()

entity = MyModel(data = {"key": "value"}, key_name="somekey")
entity.put()
entity = MyModel.get_by_key_name("somekey")
print entity.data

Hope that helps!

Update (20091126): I’ve changed db.Blob to self.data_type as suggested by Peritus in Comment. The same comment also suggested a JSONSerializableProperty subclass:

import simplejson as json
class JSONSerializableProperty(SerializableProperty):
  data_type = db.Text
  _tfm = [json.dumps]
  _itfm = [json.loads]

Thanks Peritus!

Last week, I mistakenly wrote that AppEngine Datastore would not retrieve Key fields (representing an unowned relationship) unless you annotated with defaultFetchGroup=”true”, but that it would automatically retrieve properties represented by an owned relationship. This was not correct, as AppEngine Datastore never allows joins. I’ve updated the original post accordingly.

In order to avoid JDO and/or serialization exceptions, I always call some form of persistenceManager.detachCopy() before returning objects from the Datastore back to GWT. I discovered today that you must set any non-persistent properties of the object after calling detachCopy(). My initial inclination is that detachCopy() does not copy properties annotated with @NotPersistent, but I have to do a little more testing…

Play Tic-Tac-Toe On Google AppEngine

Here’s an application I put together in Python. It is an unbeatable Tic-Tac-Toe opponent. In its current state, the human player always goes first. Just click on a square to start playing.

Click here to play

The computer’s play is based on an exhaustive analysis of positions. This is feasible since the game tree is small.

If you win, let me know. That’s a bug.

I’ve noticed that AppEngine does not populate Key fields by default when querying. I’m sure it’s documented, but I can’t remember where. At any rate, you can annotate with defaultFetchGroup to resolve this. Same goes for properties of type Text and Blob.

...
import com.google.appengine.api.datastore.Key;

@PersistenceCapable(identityType = IdentityType.APPLICATION)
public class PrayerItem implements Serializable
{
	@PrimaryKey
	@Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY)
	private Long id;
	@Persistent(defaultFetchGroup="true")
	private Key list;
	...

}

In the case of owned relationships, where a property is represented by its actual domain class rather than a Key, it is also necessary to use defaultFetchGroup=”true”. Even though owned properties are in the same entity group and are therefore eligible for inclusion in a transaction, the AppEngine Datastore doesn’t allow joins, so owned relationships are not returned in the initial query. When you set defaultFetchGroup=”true” on an owned property, you will get this warning:

WARNING: Meta-data warning for com…your_owned_property_name: The datastore does not support joins and therefore cannot honor requests to place child objects in the default fetch group.  The field will be fetched lazily on first access.  You can modify this warning by setting the datanucleus.appengine.ignorableMetaDataBehavior property in your config.  A value of NONE will silence the warning.  A value of ERROR will turn the warning into an exception.

Lazy fetching does work as described in the message.

This is the first of a series of posts that will demonstrate a variety of JDO mappings and queries with AppEngine.

For starters, let’s retrieve an object from the Datastore given its ID. Here’s the User object in our domain model:

package com.roa.client.domain;

import java.io.Serializable;

import javax.jdo.annotations.IdGeneratorStrategy;
import javax.jdo.annotations.IdentityType;
import javax.jdo.annotations.PersistenceCapable;
import javax.jdo.annotations.Persistent;
import javax.jdo.annotations.PrimaryKey;

@PersistenceCapable(identityType = IdentityType.APPLICATION)
public class User implements Serializable
{
	private static final long serialVersionUID = -1126191336687818754L;

	@PrimaryKey
	@Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY)
	private Long id;
	@Persistent
	private String firstName;
	...
}

Now we want to retrieve a User object given the ID property obtained from getId(). Easy enough, you think, as the JDO PersistenceManager provides a method for this:

// Doesn't quite work
User attachedUser = (User) getPM().getObjectById(userId);

But there’s a catch. Confusingly, the ID needed by getObjectById is a JDO ID, not the object property named “id” that we’ve annotated as an IDENTITY. You can get the JDO ID one of two ways. Whenever the object is attached, such as just after you’ve called makePersistent() on it, you can call persistenceManager.getObjectId(obj) and keep it in a safe place for later use. Alternatively, you can use a JDO helper method at any time to get the JDO ID from the object ID:

		 // Must obtain the JDO ID first
		 Object idInstance = getPM().newObjectIdInstance(User.class, user.getId());
		 User attachedUser = (User) getPM().getObjectById(idInstance);

Fortunately, if you know the object’s class, there’s a 2-arg getObjectById that does take the object’s primary key property:

User attachedUser = getPM().getObjectById(User.class, u.getId());

Of course, you can always just run a normal query on the object’s ID property, but it’s a bit ugly because the simplest query syntax always returns a Collection and we assume here that there will be only one matching object.

		Query q = getPM().newQuery(User.class, "id == :userId");
		List<User> users = (List<User>) q.execute(u.getId());
		User attachedUser = users.get(0);

Better to let the query language know that we expect a unique object:

		Query q = getPM().newQuery(User.class, "id == :userId");
		q.setUnique(true);
		User attachedUser = (User) q.execute(u.getId());

Finallly, there’s the low-level Datastore API:

		Key key = KeyFactory.createKey(User.class.getSimpleName(), u.getId());
		Entity entity = DatastoreServiceFactory.getDatastoreService().get(key);

There were about 100 folks at CloudCamp Atlanta last night. Participatns were mostly interested in Amazon EC2 or the upcoming Windows Azure platform (Microsoft was the primary sponsor). A handful of folks turned out for my “show me the code” presentation on AppEngine and GWT+MVP.

My main takeaway is that I’m really, really glad to be a Google AppEngine user. Most of the sessions and discussions centered around how to grow/shrink your Amazon server pool automatically and how to scale MySQL. These are things I don’t have to worry about at all as an AppEngine user. AppEngine is perfect for my application: launching a one-man startup that could go viral (not to mention it’s free to get started!) I’m just plain tired of sysadmin, and I’m willing to trade the  freedom to run any configuration of my choice for the scalability that comes with AppEngine’s constraints.

There was some discussion regarding the cloud and SLAs (or lack thereof). Someone said they needed five nines, not three. I don’t know where they got either number, but .999 reliability is definitely good enough for me. I have a consumer-facing app that doesn’t involve money, and I’ll be ecstatic if I have only 8 hrs 46 minutes downtime in the coming year. It’s clear that cloud offerings (especially AppEngine) appeal more to lone developers and small startups than to enterprises, although Amazon seems to have made great inroads in this area. It’s always going to be easier to start in the cloud than to move an existing data center. What excites me most about AppEngine and the like is that ONE developer can realistically create and deploy a scalable application. This has the potential to unleash a whole new wave of entrepreneurial energy on the Internet, and will make it possible for smaller companies that could never afford a data center to nevertheless get custom software for their needs. Which means we software engineers should get to do more coding and less sysadmin. Yay!

Other takeaways:

Lots of folks were happily using RightScale to manage Amazon EC2 instances and automatically grow or shrink the server pool based on load.

IBM provides Amazon machine images of its popular products like WebSphere FREE for development use (other than Amazon charges, of course).

I’m planning to do a short unconference session on securing AppEngine services with gwt-dispatch and unit testing with AppEngine and gwt-dispatch at tonight’s CloudCamp Atlanta.

I’m looking forward to meeting some local AppEngine developers, as I’ve been feeling awfully close to the bleeding edge lately. I routinely find that solutions have been posted on AppEngine forums just eight days ago, and sometimes don’t exist yet. I really need TaskQueue in order to send out emails, which is still experimental in Labs, and would even more like to use deferred.defer (less than two weeks old), but alas, it’s currently only available for Python.

The way it stands now after my last several posts on unit testing is that each test method in a JUnit TestCase runs its own context, including its own PersistenceManager injected with Guice. Because I’m initializing the AppEngine test environment with LocalDatastoreService.NO_STORAGE_PROPERTY = TRUE, any test data I create in one test is not available to the next test method, even in the same TestCase class. This is typical behavior for JUnit test cases, and generally a good thing as each test should be independent, but it does mean we need a convenient way to create test data. My current strategy is twofold:

1. Create test data used by all tests in the TestCase setUp() method that runs before each test
2. Create common test data (used by multiple TestCases) in test data generator services

The first problem to solve is how to get access to a dispatch service and PersistenceManager inside the test data generators. Both are now being injected by Guice as covered in previous posts, so I’ve written a BaseTestDataService that does the Guice magic:

package com.roa.test.service;

import net.customware.gwt.dispatch.client.standard.StandardDispatchService;

import com.google.inject.Guice;
import com.google.inject.Injector;
import com.roa.server.guice.ServerModule;
import com.turbomanage.gwt.server.PMF;
import com.turbomanage.gwt.server.guice.DispatchTestModule;

public class BaseTestDataService
{
	protected static Injector inj = Guice.createInjector(new ServerModule(),
		new DispatchTestModule());;

	protected static StandardDispatchService getService()
	{
		return inj.getInstance(StandardDispatchService.class);
	}

	protected static PMF getPMF()
	{
		return inj.getInstance(PMF.class);
	}
}

I’m intentionally using a static getPMF() method to make it as simple as possible to use the test data services in a unit test via static methods. Alternatively, I could have used instance methods and constructor injection, but then you’d have to create a new instance of a test data service in each test, which is just that much more code to write… Also, constructor injection is not possible in this case because the TestCases themselves are not instantiated by Guice, so instantiating a new test data service from a test case would not involve Guice, either.

It does not matter that the BaseTestDataService and BaseTest (below) are both calling Guice.createInjector() and thereby creating multiple Guice contexts, each having its own PMF instance. The important thing for these database tests is that they’re all going against one Datastore, not one PersistenceManager.

Here’s a simple test data generator that extends BaseTestDataService and provides a method to add a test user:

package com.roa.test.service;

import com.roa.client.domain.User;
import com.roa.shared.rpc.AddUserAction;
import com.roa.shared.rpc.AddUserResult;

public class UserTestDataService extends BaseTestDataService
{
	public static User addTestUser() throws Exception
	{
		// Create new user
		User u = new User();
		u.setEmailAddress("test@example.com");
		u.setFirstName("Test");
		u.setLastName("User");
		u.setGoogleAccountId("testAccountId");
		AddUserResult userResult = (AddUserResult) getService().execute(
			new AddUserAction(u));
		u = userResult.getUser();
		return u;
	}
}

Note the call to getService() on line 17. Thanks to Guice, test data generator services can invoke gwt-dispatch ActionHandlers the same way as the tests themselves.
Finally, here’s a test case that calls the above service to create a test user in the Datastore:

package com.roa.test;

import java.util.List;

import javax.jdo.Query;

import com.appenginefan.toolkit.unittests.BaseTest;
import com.roa.client.domain.User;
import com.roa.test.service.UserTestDataService;

public class UserTestCase extends BaseTest
{

	@Override
	protected void setUp() throws Exception
	{
		super.setUp();
		UserTestDataService.addTestUser();
	}

	public void testUserAdded() throws Exception
	{
		// Run tests here
		Query q = pm.newQuery(User.class, "emailAddress == e");
		q.declareParameters("java.lang.String e");
		List<User> users = (List<com.roa.client.domain.User>) q.execute("test@example.com");
		assertEquals(1, users.size());
		User user = users.get(0);
		assertNotNull(user.getId());
	}
}

I should probably show my BaseTest method, too. I’m using AppEngineFan’s BaseTest as discussed in previous posts and modified the setUp() method for Guice injection as follows:

	/**
	 * Sets up the App Engine environment.
	 */
	@Override
	protected void setUp() throws Exception
	{
		if (initializer != null)
		{
			throw new UnsupportedOperationException(
				"setup may only be called once!");
		}
		super.setUp();
		initializer = new TestInitializer(getEnvironmentOrNull());
		initializer.setUp();
		Injector inj = Guice.createInjector(new ServerModule(),
			new DispatchTestModule());
		this.testSvc = inj.getInstance(StandardDispatchService.class);
		this.pm = inj.getInstance(PMF.class).getPersistenceManager();
	}

Thanks to Guice, we can now get access to the PersistenceManager and call ActionHandlers via a dispatch test service even from static methods in test data generators. This greatly streamlines unit testing.

Regarding my previous post, it turns out that I needed a TestPMF implementation sooner than I thought. The reason is the way I’m injecting a DispatchTestService in my unit tests (as described in this post). I’m calling createInjector() in the setUp() method, which gets run before each test method:

	@Override
	protected void setUp() throws Exception
	{
		super.setUp();
		Injector inj = Guice.createInjector(new ServerModule(),
			new DispatchTestModule());
		testSvc = inj.getInstance(StandardDispatchService.class);
		// This pm is needed only for code in this class that calls a PM directly
		// ActionHandlers are injected with the PMF singleton from Guice
		pm = inj.getInstance(PMF.class).getPersistenceManager();
	}

Since setUp() gets called before each test method, Guice is initialized for each test method, and therefore Guice calls the constructor for my DefaultPMF class for each test method. Repeated from the previous post, the DefaultPMF class looks like:

package com.turbomanage.gwt.server;

import javax.jdo.JDOHelper;
import javax.jdo.PersistenceManager;
import javax.jdo.PersistenceManagerFactory;

public final class DefaultPMF implements com.turbomanage.gwt.server.PMF
{
	private final PersistenceManagerFactory pmfInstance = JDOHelper
			.getPersistenceManagerFactory("transactions-optional");

	public DefaultPMF()
	{
	}

	@Override
	public PersistenceManager getPersistenceManager()
	{
		return pmfInstance.getPersistenceManager();
	}
}

Because DefaultPMF creates a named PersistenceManagerFactory, JDO complains that the named PersistenceManagerFactory has already been created. My solution for now is to replace the DefaultPMF with a TestPMF that uses a Properties map to initialize the PersistenceManager, just as the AppEngineFan TestInitializer does. Here’s a working TestPMF:

package com.turbomanage.gwt.server;

import java.util.Properties;

import javax.jdo.JDOHelper;
import javax.jdo.PersistenceManager;
import javax.jdo.PersistenceManagerFactory;

public class TestPMF implements PMF
{
	private final PersistenceManagerFactory pmf;

	public TestPMF()
	{
		Properties newProperties = new Properties();
		newProperties
			.put("javax.jdo.PersistenceManagerFactoryClass",
				"org.datanucleus.store.appengine.jdo.DatastoreJDOPersistenceManagerFactory");
		newProperties.put("javax.jdo.option.ConnectionURL", "appengine");
		newProperties.put("javax.jdo.option.NontransactionalRead", "true");
		newProperties.put("javax.jdo.option.NontransactionalWrite", "true");
		newProperties.put("javax.jdo.option.RetainValues", "true");
		newProperties.put("datanucleus.appengine.autoCreateDatastoreTxns",
			"true");
		newProperties.put("datanucleus.appengine.autoCreateDatastoreTxns",
			"true");
		pmf = JDOHelper.getPersistenceManagerFactory(newProperties);
	}

	@Override
	public PersistenceManager getPersistenceManager()
	{
		return pmf.getPersistenceManager();
	}

}

Now simply bind PMF to its TestPMF implementation in your Guice module for tests (DispatchTestModule in the setUp() method above), and each unit test method will run with a freshly created PersistenceManager.

This blog is about my road to become a multi millionaire web entreprener, dreaming is free . I’ll be writing lots of technical stuff, like GWT (Google Web Toolkit), Google AppEngine, Amazon EC2, Java, ruby, html5, css, linux or whatever topic I think is interesting. But also there’s going to be post about <insert your topic here>, which is really great! Haha, I say this because I don’t want to feel my hands are tied up because of some silly structure.

That being said, “who the hell are you?” you may ask. Well my name is Francisco Ceruti, I’m a college student who’s biggest dream is to be “the most successful drop out”. Bill Gates said that phrase on a speech he gave to the Harvard graduates (you may guess, and correctly, that I’m a youtube junky), and I thought that was the biggest honor I could have, so right now, that’s my personal goal.

Please watch the following add:

I thought it was hilarious, and the reason I’m showing you that, is because I have someone I look forward to a lot, he is not as cool as the usb port inventor, but anyway, that guy is Steve Jobs. The man leads a team that makes elegant products, with really polished user interfases, cares about how it looks and feels in your hands, basically takes into account so many more factors than what the competition in general does. He really sets as a first priority the user experience. I’m not a mac user myself, mostly because they are so expensive, but if I had the money I would undoubtly have at least one. For now, I’m happy with Ubuntu (psss, 3 days for 9.10 Karmic Koala release).

I’ll try to submit new post on a weekly basis, and hopefully you will enjoy it. Also, I resently got twitter, so if you like you can follow me! Mi nick is fceruti.

PD:One thing I want to add is that my native language is not english, nor I live on a english speaking country, so any help with that would be greatly appreciated.

This is a follow-up to last week’s post on unit testing ActionHandlers with Guice. David Peterson pointed out on the gwt-dispatch mailing list that I could inject a PersistenceManager into my ActionHandlers in order to provide an alternate PersistenceManager for unit testing. I don’t actually need an alternate PM yet as it is handled transparently by my AppEngine test environment, but I thought it would be easier to do it sooner rather than later, so here goes.

I’ve created an interface called PMF (in order to avoid confusion with JDO’s PersistenceManagerFactory).

package com.turbomanage.gwt.server;

import javax.jdo.PersistenceManager;

public interface PMF
{
	PersistenceManager getPersistenceManager();
}

My default PMF implementation works exactly the same as before:

package com.turbomanage.gwt.server;

import javax.jdo.JDOHelper;
import javax.jdo.PersistenceManager;
import javax.jdo.PersistenceManagerFactory;

public final class DefaultPMF implements com.turbomanage.gwt.server.PMF
{
	private final PersistenceManagerFactory pmfInstance = JDOHelper
			.getPersistenceManagerFactory("transactions-optional");

	public DefaultPMF()
	{
	}

	@Override
	public PersistenceManager getPersistenceManager()
	{
		return pmfInstance.getPersistenceManager();
	}
}

The DefaultPMF gets bound in my Guice ServerModule:

	@Override
	protected void configureHandlers()
	{
		...
		bind(PMF.class).to(DefaultPMF.class).in(Singleton.class);
		...
	}

And finally, the PMF is injected into an ActionHandler:

public class AddUserHandler implements
	ActionHandler<AddUserAction, AddUserResult>
{
	@Inject
	private PMF pmf;
	private PersistenceManager pm;

	@Override
	public AddUserResult execute(AddUserAction action, ExecutionContext context)
		throws ActionException
	{
		this.pm = pmf.getPersistenceManager();
		...
	}
	...
}

Now, when the need arises for a test implementation of PMF, I can easily bind a different implementation in my Guice TestModule as shown in the earlier post.

Update: I did not test this thoroughly before I posted, and the need for a TestPMF arose just hours later. See my next post for the solution.

Image via CrunchBase

I have readopted JDO recently. The cause ? Google App Engine’s big table storage model that makes JDO, a storage agnostic framework for persistence, ideal for such implementations.

Some very useful links related to Java on the GAE are following

1. JRE Class White List A Java App Engine application’s access to the classes in the Java standard library (the Java Runtime Environment, or JRE) is limited to the classes displayed in that link.
2. Will it Play in App Engine. This google group post aggregates all known java technologies and frameworks with commenting on whether or not they are applicable to the appengine JVM runtime. As we have discussed before limitations to certain JEE classes will stop many of the well adopted frameworks to operate.
3. Web Services Connector. Due to limitations in the java.net core library many web services client frameworks fail to operate in appengine. Frameworks such as axis, axis2, JAX-WS do not have compatibility since they use not authorized classes. A simple and lightweight alternative is   WSC from people in salesforge.com that generates a simple jar client provided the wsdl document services definition. Another more nasty alternative is to implement the calls to the service by yourself and use JAXB or JIBX or even xmlnbeans to marshal/unmarshal the request/response documents.