Wiring up Model-View-Presenter UIs using Google Gin

We make heavy use of Google Web Toolkit in our dashboard systems: Analytics, management, configuration, and so on. In order to make these applications clean, testable, and modular, we apply the Model-View-Presenter pattern. However, MVP alone is not a particularly good pattern – views are tied to presenters by a controller class and have a circular dependency with their presenters. Changes to the system can require altering multiple moving parts.

This is where we’ve applied Google Gin as a way to create a more loosely coupled implementation of MVP. We still have to put up with the view/presenter circular dependency, but it is cleaner and more manageable now.

Here is how:

First, the GWT module initializes the dependency injection cascade by asking an injector we wrote for the controller class:

    private final AppGinjector injector = GWT.create(AppGinjector.class);
    public void onModuleLoad() {
        injector.getDashboardController();
    }

Read the rest of this entry »

Refactoring away Singletons with Google Guice

We’ve been engaged on a number of projects lately, such as integrating the and jsTestDriver on the front-end and rolling out Guice and ASM on the back-end.

One of the most enjoyable aspects of these projects has been refactoring the Singletons in our system: We see the typical implementation of Singletons as an anti-pattern that creates tightly bound systems with tangled dependencies.

Rather than showing our own code, I wrote a simple project to illustrate one way we get rid of Singletons. The Main function starts up a pair of Singletons, lets them do some work, and prints out the details:

        EventListenerService.get().initialize();
        EventGeneratorService.get().start();
        Thread.Sleep(100);
        for(Event e : EventListenerService.get().getRecordedEvents()) {
                System.out.println(e.info);
        }

Read the rest of this entry »

On Layering

While we only sell a single product – our Social Commerce suite of applications, internally, we’ve had to build a number of commercial-grade systems, some of which could be seen as products in and of themselves.

Product 1: First of all, in order to deliver components into remote websites at massive scale, we’ve had to build a sophisticated platform that can deliver content across the web, and integrate data from multiple websites while remaining cross-browser compliant. This platform is built on Java, AWS, CloudFront, Memcached, and other sophisticated “shoulders of giants” systems.

Product 2: Next, in order to deliver analytics over these components, which can be loaded thousands of times per second, we’ve had to develop an analytics platform based on Pig, Hive, Hadoop, S3, and Postgres. This is a layered system, providing multiple tiers of services. We’ve also had to develop our own link shortener.

Product 3: The most visible of our products is our suite of social applications. These build on top of the other two products to deliver trackable, scalable, personalized social content to users. This system, built using JavaScript, YUI, Freemarker, Java, Ehcache, Memcached, and Postgres, sits on a complex multi-tenanted graph database that tracks social graphs and provides deep personalization data.

By treating each part of the system as a standalone product, and building each one to the highest level of performance and API quality, we ensure that new products can be rapidly built by leveraging existing technology investments, which allows us to innovate at a business level – where, at the end of the day, it really counts.

On Shortcuts

Startups are often kick-started with a prototype. Perhaps not much more than tape and baling wire, this prototype can get the company users, noticed, funding.

Success is when the moment of true danger is, and the test of top-notch engineering and product leadership: Success is when you need to grow and evolve your system.

If you have the luxury to throw away your initial code and start from scratch, you’re in an unusually good position. If you aren’t, then the decisions made at the prototype stage have a profound impact on your forward velocity. A badly written prototype, full of cut corners and shortcuts is going to severely impair your ability to write new features, to find and fix bugs, to write a world-class test system.

The key is to know where to take shortcuts, and this applies even beyond the prototype stage: Never, ever take shortcuts on architectural correctness or code quality. Take shortcuts on features, on configurability, on performance. Don’t make every part of your system configurable, but architect it so that it can be. Don’t try to write a massively complex sharding system – rely on a single, robust database with a hot replica. Don’t attempt to give users three ways of doing anything in your system; focus on the path of least resistance.

At each step of the way, build just enough code to validate assumptions and test ideas, but build it right.

On SaaS and the Law of Large Numbers

Among our design mantras is that everything that can go wrong, will go wrong.

Is the concurrency model not perfect? Under the onslought of 10,000 hits per second, it’ll break.
Is the caching model not thought through completely? It too, will break.
Is there a single point of failure anywhere in the system? It will bring your system down.

We’ve learned to be, and to hire, perfectionists, because shortcuts do not work in a sophisticated, high volume environment. We’ve learned that every solution must be thought through and peer reviewed because any sufficiently complex system cannot be fully understood by a single person. We deploy at least two (typically four) instances of anything.

We are moving towards a model where the system is so loosely coupled that we’ll be able to shut down every single one of our database servers, and have the system continue running with no visible interruptions or loss of data. We can already shut down any web server with no visible effect to the system, as well as seamlessly launch new instances.

We’ve learned to rehearse, to “dry run” all significant operations, and to challenge each other to approach the theoretical limits of operational goodness – to have zero downtime even for significant system changes.

All this requires very different thinking from traditional enterprise systems. It is even different from running traditional high-volume websites, because we are responsible for very different websites across far flung, geographically diverse domains, each with highly isolated data stores running on a shared fabric.

We love the enormous amounts of learning this involves, the need to constantly improve ourselves and our systems to handle constantly increasing loads. If you’ve built a high-scale SaaS system and evolved it over time into a best-of-breed solution that is truly stable under enormous load spikes. If you’ve learned to be a real engineer under these conditions – to understand how to solve problems thoroughly and deeply and to consider issues of monitoring, scale, failure, analytics, concurrency, availability and consistency, then there are few systems you cannot be confident that you cannot build.