Comping in the cloud is confusing

Around a month ago I posted about a c# winforms application I wrote to crawl a comping forum (a forum dedicated to free-to-enter online competitions). The goal was to present me with a list of available competitions and let me enter them all from the one place.

The little app I wrote worked nicely and did what I wanted, but it had it's flaws.

Firstly, it was slow. Any time I wanted competitions to enter I had to ask it to crawl through numerous pages and then parse everything there and then. I had to wait for it to do this every time before I started "comping".

Secondly, it had no concept of state. It would crawl the same pages again and again, every single time I launched the app. It would thus eat my bandwidth, the forums bandwidth, and most times I'd just see the same contests appearing again and again anyway. 

Lastly, it was platform dependent.  It was a winforms desktop app, so I could only use it on a Windows Desktop PC with the right version of .NET installed and yadda yadda so on and so forth. Microsoft appear to be backing away from the Windows desktop these days, so it wouldn't be very future proof.

Enter cloud services. 

Cloud services, as I have recently discovered, offer a way of placing the business end of my program logic onto a server somewhere, so that it can do all the hard work in the background. The user can then simply use their dumb-client phone app or similar to access cached results quickly and cheaply. This is probably a horrid over simplification, but that's what a cloud service is as currently relates to me.

This offers me a way of placing the logic to crawl competition websites into a long running task on somebody else's server and have it save the results to a database. It offers me the ability to pull that information down in the  JSON format in a way which means my bandwidth is unaffected. This also means that I could write an iPhone or Android app to similarly retrieve that data.

Cloud services are hard. 

Ok, cloud services aren't really that hard, but they are pretty damn tricky when you don't know what you're doing. My language of choice is c#, so I had the options of WCF, Web API, ServiceStack or doing it manually with a port listener. Up until last weekend, I had very little idea what any of that meant.

WCF is great in theory; it's self documenting and using it with c# and Visual Studio is easy peasy, it scales well, and I had used it briefly before at University. It isn't great at cross platform integration though, or at least, I couldn't work out how you'd do that, so in the end I opted for ServiceStack.

ServiceStack is a web service framework for c# and is a dream come true in terms of what it offers. It's scalable, does most of the heavy lifting for you, and is easy to get up and running with. It's all in the configuration apparently. I'm still guessing with most of my web.config configurations, but I'm hoping I'll get there sooner rather than later. I followed the tutorial at the ServiceStack HelloWorld tutorial which got me up and running. I modified the code a bit to hook into my own CompetitionCrawler code, and what I have at the end is a service that is set up like this:

public override void Configure(Container container)
	//register user-defined REST-ful urls

	//Register all your dependencies
	container.Register(new CompetitionContext());	

and is defined like this:

public class Competition
    public int NumComps { get; set; }

public class CompetitionResponse
    public List<inkItem> Result { get; set; }   
    public CompetitionResponse()
        Result = new List<LinkItem>();

public class CompetitionService : IService<Competition>
    public object Execute(Competition request)
        return new CompetitionResponse 
            Result = CompetitionCrawler.GetCompetitions(request.NumComps)

This code sets up the requisite routes, such that I can call the service with an int, which specifies the number of competitions to return. My service takes that int, passes it to the CompetitionCrawler class which queries the database off camera and populates the result property of the CompetitionResponse object. ServiceStack then takes care of the rest and lets me retrieve that response object in XML, JSON or a number of other related cross platform formats.

Easy peasy when you know how!