Direct YouTube Link: http://www.youtube.com/watch?v=peYSjTxnzK0
Direct Vimeo Link: http://www.vimeo.com/12230792

Cost-Effective Scalability, Oh-my!

Lightweight Models & Cost-Effective Scalability, Oh-my!

Lightweight Models & Cost-Effective Scalability, Oh-my!

It might seem like a quest to have the best of everything but Lightweight Models & Cost-Effective Scalability isn’t that mythical. You don’t have to battle the the wicked witches, but that doesn’t say it is a simple path to take either.

So lets talk about an example that exists out there already:

Wikipedia – www.wikipedia.org

We all know what Wikipedia is but in case you missed it (perhaps your just starting to use a computer?), Wikipedia is a free-content encyclopedia that is based on an openly=editable model. It allows users to add articles/pages and also edit existing ones in a self organising way.

So lets take a look at some stats and see how large Wikipedia is, bear in mind this is only the English side of Wikipedia and doesn’t count for other languages.

I’ve also put together a more visual representation taken from the wiki stats page, Click to enlarge the view!

So there is no doubt that Wikipedia is large, and they have use an Open Source, Scalable solution across multiple servers, multiple countries, etc. All with less than 30 staff members dedicated staff members. Of course you don’t have to be as large as wikipedia to use the software and there are in fact many smaller sites that use the same software. You might not have the large economies of scale either, and that doesn’t matter because there are effective solutions out there.

Enter LAMP, a solution stack of open source software that is ready to take on your needs. LAMP is actually an acronym (not a light stand) that represents the key open source software packages.

Linux – An operating system
Apache – A HTTP web server
MySQL – Database software
PHP – A language in which you can code web pages

LAMP allows you set up and host your own site with ease, it is scalable and as for the cost effect side – it is Open Source. I’ve heard lots of rumours but I haven’t been able to find any ‘hard facts’, but major sites like Facebook and Flicr are based on LAMP, or at least were at some stage. I could say my site is based on a LAMP distro to also make a point (but it isn’t), however there is nothing stopping you from starting your own site and doing so.

The next biggest craze though is cloud-computing and business creating their sites of API’s, and renting CPU power from Google, while hosting data on Amazon S3, etc. The reason I avoided this is because it is very much on the hype cycle and I don’t feel comfortable not having that much control. This might be changing (and it looks so), but it is hard to recommend outsourcing a solution when you can get started yourself easily with LAMP!

The long tail describes a wider range of products that are more specific getting sold less often, rather than the ‘head’ being more common/generic products being sold more often. A great image create by leftclick can be seen below:

Effectively the dinosaur represents a curve on the graph, where there more generic or common a product, the increase of the results. On the other end the more specific the term, the less results occur. Following the example by leftclick, how would you leverage the long tail in web 2.0?

If you were in the hotel business and you had a resort you could describe yourself as a ‘hotel’ or perhaps even the location your hotel was. This would be a very generic term, and really ‘the head’. If you were more specific, you could use key works such as ‘accommodation’, ‘Queenstown’, ‘backpacker’, ‘wheelchair access’, ‘pets allowed’ and the results would be fairly narrow. Putting this keywords into Google may give you the results of a few million (the head), to a few hundred or less (the tail).

The more specific your key words, the more specific your audience – leveraging the long tail can yield lower costs in advertising while also increasing your conversion rate.

By doing the following you can target advertisements on the long tail in the example above. This also means you will have lower costs (because there is less competition) and have higher conversion rates (because the target audience is looking for services similar to what you offer).

Another great tip is from leftclick is focused pages, targeted at these specialised search terms. What this basically means is, don’t rely on your advertisements bringing in visitors, create pages or content so the search engines results will list the page (in in turn bring them to your site). In the above example creating an online guest book, or encouraging user reviews (even better if they blog about it and link to your site) are a few good ways to create content.

So what if your a site like Amazon? Similar approach works but there is a few more tips which you can use. If you have a large product range, once the visitor is on your site you might want to direct them towards more generic items you have for sale. The products are more popular and as a first time vistior you might be more interested.

Perhaps you’ve become a member of a site like Amazon and bought a few items. Soon you wont to direct your members towards a little more ‘specific’ material. This will have the opposite effect in that existing members who have seen the common/generic stuff can get pushed towards the more specific terms. By doing this you are encouraging existing users to branch out more.

Perpetual – perpetually – In a perpetual manner; Seeming to never end; endlessly; constantly
en.wiktionary.org/wiki/perpetually

Beta – A software release is the distribution (whether public or private) of an initial or upgraded version of a computer software product. The software engineers and company doing the work decide on how to distribute the program or system, or changes to that pre downloads and compact discs.
en.wikipedia.org/wiki/Beta_(software_development)

Perpetual Beta – Perpetual beta is a term used to describe software or a system which remains at the beta development stage for an extended or even indefinite period of time. It is often used by developers in order to allow them to constantly release new features that might not be fully tested.
en.wikipedia.org/wiki/Perpetual_beta

Perpetual Beta in Web 2.0 is used to describe an online service which remains under constant development. At great example of this was Gmail.

Google’s Gmail service launched on the 1st of April, 2004 as an invite only service. I remained as an invite only service for nearly 3 years before becoming open to the public (7th of February, 2007). It was still labeled a Beta for another 2 years until the lifted the Beta labelled along with the rest of the Google Apps Suite on the 7th of July, 2009.

During its time in Beta Google constantly introduced new features and changed its services. Introducing new languages, enabled IMAP support, increasing the mailbox size, integration with Google Docs, custom themes, Google labs and many more. Click here for a history of Gmail.

So why label something a Beta? Generally if something is in Beta you can expect some things not to work, and problems arise. This is all part of harnessing the collective intelligence to develop the product, find bugs and provided feedback to make a better product. Without having a web 2.0 application or service you will be stuck in the traditional software life cycle which can cost more, take longer and not be as effective.

So why take the Beta label off after so many years? From an Infrastructure perspective you don’t deploy products that are in Beta. You deploy products that a ready that have no expected bugs or problems. Google understood this concept so after around 5 years the finally took the Beta label of. This changes how a business can look at a product and it will allow business to evaluate Gmail and related apps as a potential solution. Previously this would have been ruled out straight away because it wasn’t ready, it was in ‘beta’ and a business can’t afford to have systems that are un-reliable.

So while the main service of Gmail isn’t in Beta anymore there are still more features on going. Google introduced Google Labs which is an opt-in only service that lest to add features and functionality that are in a ‘Beta’ phase. I actually use some of these myself such as multiple inboxes, offline synchronisation, youtube preview in e-mails just to name a few. Once these have been tested enough they might make it to the core functionality of Gmail. So this is an example of a Web 2.0 service that is on a constant Perpetual Beta.

For a user to receive data from through the internet there is a least software running on the servers and and software running on the clients end to get this to work, however I don’t this this is the point Tim is trying to make. I like to think of the concept as software allowing you to do more with the device, than if the software was isolated.

Take for example the Foxtel’s IQ cable box. It allows you to view channels, record shows, bring up an EPG all on the device. This functionality is extended through an iPhone App or their website which allows you to program your IQ box through the internet. It has extended your experience from using the software on the device itself, to using web software/ application to view the EPG, schedule in shows to record without using the IQ box directly.

This has extended to other areas inside your home, take a look at the gaming industry. Sony has create an online community, integrated with its products called the Playstation Network (PSN), Microsoft has done the same with X-Box Live and Nintendo with their online community (although very limiting).

I’ve been trying to stay from examples such as Google Maps, E-Mail, Search, Images, Docs, etc simply because it there is so much to talk about and how their software through API’s, Common Protocols, Applications, etc allows their own software to be used on a wide range of devices, but really if you look at any of them you could drill down and see how they are examples of software above the level of a single device. In fact, Minterest.com published an article recently called ‘Over 60 Google Products & Services You Probably Don’t Know‘ that gives you a break down on what Google offers. There is actually a fair few I didn’t know about and haven’t used.

Another similar example of been trying to escape is how the iPhone connects into iTunes on your computer. This then allows you to configure the device, sync selected data such as photos, music, playlists, applications, bookmarks, etc. While iTunes is allowing you to download music, update software on the phone, subscribe to podcasts, add lyrics to songs, download album artwork, etc all without using the iPhone itself. This can all then can synchronised back to the phone. Apple have then added in another software component called mobile.me which could deserve a post in itself (Take a look at Anthony Nicolaides’s post on ‘MobileMe above the level of a single iDevice‘ to find out more).

Looking back on how far we have come in nearly three years. There is still a lot more companies can achieve by providing ‘software above the level of a single device’ as there a still plenty of gaps above. With cloud computing increasing in popularity and continued progression of technology it really is becoming an easier world to live in. All we need now is for companies to not lock themselves into a ‘walled garden’ and the ease of synchronising myself to the digital world will become second nature (Wow! I feel like a holiday, looking at how much I use technology in my daily life.. perhaps I’ll Google holiday resorts on my iPhone).

The reason you revisit a website is normally because it has updated, there is new data, you have found a purpose to return and all of this is intended to generates a rich user experience that makes you want to return. There are however ways of getting ahead of the crowd and making that user experiences better.

10 years ago you might have been tempted to use animated graphics on a page with some fancy DHTML coding to make you think ‘wow, that’s cool’, however technology has advanced and so should your website. Today a site will use AJAX, PHP, .NET, HTML 5 to create a richer experience for the user. These standards allows a developer to load content that refreshes automatically, embed videos, images, upload data, etc.

The next level up is creating a Rich Internet Application. Now you might be thinking application in the sense of a program you find under the menu in your computer but I’m talking about a Rich Internet Application that uses Adobe Flash, Java or Microsoft Silverlight (the three biggest players in the area). Before you can actually view something in Flash, Java or Silverlight you will need to download and install the plug-in. However once installed you will be able to download a small application on the website and run it in your browser. There are however downfalls in going this way. Silverlight doesn’t work in Linux, Flash doesn’t work on most mobile devices, Java can be problematic on some PC’s, etc which need to be taken into consideration.

So lets move on to some examples! There are plenty out there so I’ve just decided to to pick an example site from Adobe Flash, Java and Microsoft Silverlight:

Adobe Flash – Donnie Darko (the movie)

This one I might be a little Biased to as I really enjoyed the movie and when I discovered the website I was impressed. The website is based on flash and is a great example of a RIA (Rich Internet Application). It combines, text, video, audio and takes the user on a journey through the website. In fact it is a challenge the whole way through as information is provided through to you in short bursts, mouse clicks are hidden at times and you are prompted to put in passwords to open the next section (should you be clever enough to work it out from the information provided).

Java – Yahoo! Games

Yahoo! Games are based on Java. I’ll rephrase that. Most of Yahoo!’s games that you play online use Java. They have a wide range from poker, scrabble, pool to Bejeweled. This has created an incredible series of RIA which gives the user a Rich User Experience that keeps them coming back! I would suggest to give it a try, but I warn you that you may become addicted.

Microsoft Silverlight  - Silverlight.net Showcase

Silverlight is the newest of the three, however this doesn’t mean it hasn’t got any attention. In fact Stat Owl, has showed that Silverlight is slowly growing in market share. The showcase site itself is in in Silverlight, but it provides many examples (over 900) of websites that are using Silverlight. These range from MMORPG, public transport, geospatial imagery to dynamic bar graphs.

Let me know if you have any stand-out examples of Rich Internet Applications!

Twitter, the online social networking site is a great example of an innovative company embracing this technology. Although their premise is simple (allowing its users to post text-based messages of up to 140 characters long), they have adopted APIs, which allow other web services and applications to integrate with their site.

Some examples of applications and other sites that have used API’s twitter have provided are as follows:

Another Tweet On The Wall – Enter in a Hash tag it will start to display random twitter posts in a nice flash page one at a time. If the user has posted a link to a picture (also through an API), then it will load the image and display it as the background.

Twitter Updates – I totally forgot, but on my site you can see that there is a section on the side which show my latest twitter posts. It pulls the data through the use of twitter’s API’s.

Twitterrific - Desktop application for the Mac which lets you use all of twitters features, without ever needing to go to the original site. They also provided an application for the iPhone which does similar things.

In fact, there are far too many examples to write down here so if you are interested check out the following sites which have more examples, and tutorials on Twitter API’s.

• Smashing Magazine – 50 Twitter Tools and Tutorials for Designers and Developers
• Twitter API Wiki – The official Twitter API wiki, including resources, helps, examples and further information.

The use of API’s in Web 2.0 provide unforeseen opportunities to the core website. It allows 3rd party developers to use the data inside the website and extend the features, functionality in creative and innovative ways.

Google is one example of a service that has transformed dramatically over the last decade. It primarily began as a unique way for consumers to search through the Internet using special text-matching techniques. However, over the years it has grown to encompass other technologies and services, such as e-mailing, online mapping, social networking and video sharing. These new services have only been made possible by taking advantage of Google’s enormous range of data collected and stored over the years.

Above is an image I produced using TouchGraph.com doing a search for ‘google’, which visually displays the major links associated with Google.com. You can try it out yourself!

The Google information database is extremely valuable, with each new web application heavily reliant on this information. Effective database management has become an absolute necessity for the company and the data it holds has become extremely valuable. With out this data, Google’s services wouldn’t exist. This is also a major challenge for any new comer to the market as they will be starting off with an ‘empty’ database as such.

As Web 2.0 sites grow and develop, their data will become increasingly important. Websites such as Facebook, Twitter, YouTube and Wikipedia place less emphasis on providing content themselves, and a move to user-generated content. There is a far greater focus on the conservations between users, as opposed to providing a place for advertisers to promote brands. Between 700,000 and 1.3 million blogs are made daily and according to Google, a new blog post is being created every second of every day (James, 2009).

There have been many ethical and legal questions raised regarding the website’s privacy settings. Facebook has been featured on many news programs both locally and internationally, with people questioning whether its privacy settings are indeed safe and confidential for each user. Clearly, being in control of this large range of data is a huge and challenging responsibility.

Since many Web 2.0 services thrive on the participation between users, data storage services become more valuable. Consumers depend on these sites for a safe, fun and interactive environment to connect with other people and information. If the Web 2.0 site fails to provide this service, then the demand will reduce. It is becoming more apparent nowadays that the data found on a site is just as important as the actual software. Google has recently shown this to be the case with their investment to DigitalGlobes imagery and geodata. As technology progresses, we demand more than what the Intel Inside chip has to offer. People now have an interactive environment, where communication is fast, simple and reliable.

James, L April 2009, ‘Should you advertise on social networking websites?’, World Advertising Research Centre, viewed 10 July 2009

• Blackboard @ QUT
• Facebook Community
• Book a consultation slot with Jason
• Lecture Audio / Podcast Feed (you can subscribe to the podcast stream or right click and save individual files)
• Blog index for INB Undergraduate students
• Blog index for INN Postgraduate students
• Aggregated / Blog Feed? *need to find or create one from all blogs*

Perhaps a more familiar term is ‘The wisdom of the crowd’. The idea is that many people can do more than one, a collection of opinions can construct a better opinion than that of an individual. For example, if you had a problem with the brakes on your car but weren’t sure why you could ask someone. They might respond that it could be they need replacing, while if I ask another person they may suggestion to check the brake fluid in the car. Another person may give another solution, and before you know it you have a list of 5 things to look at which is more helpful than before. You have started to hardness a collective knowledge and have a much better opinion than if you had of asked one person.

Continuing on with this metaphor, problems can happen. The people I might have asked could be all Holden mechanics and showed a bias against my Ford. The range of people who I asked might be very wide – as in some opinions were very qualified as they have been an mechanic for 30 years and another person may have never drove a car before yet their opinions get valued the same (as I might not know there background).

So how can this be applied to the internet? Are there any examples?

Wikipedia – It hardness collective intelligence as multiple people can contribute and edit articles to provide better answers and once that are more up to date. In many cases it does better than many encyclopaedias.

Yahoo! Answers – A way to ask questions and find answers. You can post a question and then people can answer (similar to my example above).

E-How – A place where you can find how to do something, or tell the world how to do something you know. From E-How’s home page, “What would you like to do? How to Attract Consumers? How to Lose Weight? How to Make Money in a Bear Market? eHow is the world’s most popular place to find step-by-step instructions on how to do just about everything.”

So is anything on the web an example of collective intelligence?

No, information must be presented or processed in some sort of way, otherwise it is a collection of opinions and not knowledge. This information must flow freely and not be censored.

The collective or group needs to also have some attributes to. Opinions to be as diverse and independent of one another (a persons opinion shouldn’t be a reflection of someone else opinion), aggregation (a way of hardness opinions to make a decision or judgement).