Short.ie, social urls, the idea, the features

Posted by David in echolibre, industry, innovation
Monday, December 8th, 2008 at 21:19

What do Digg and short URLs have in common? They are URLs, they are shared and used by many people. Why not mixing the idea of short URLs (Which has mainly arised because of the Twitter hype) and the ability to get people to share and publish URLs to the public like Digg?

So a few months ago I was discussing with a few lads about all this, and we decided to start on our own social network idea, with URLs. Basically one thing we noticed is that many many of the url shortening services had cool features like stats, custom urls, and at the same time you saw the power of Digg’s public URL hype idea. We basically decided that if we could make an URL shortening service that had most of the features of the other services in one place, and that if we could also add some value to this by bringing a “social-networking” aspect to it, it would simply rock!. Instead of linking people by their type of types of people, activities, technology interests, groups of friends, etc. we saw that there was an opportunity to link people by the type of urls they shorten but most importantly the context of those URLs.

Right now as it stands, you can have public or private urls with short.ie. When you shorten an URL and make it public, we basically parse the content of that page, analyze it’s title, description, headings tags and a few other things (wink). We then store all this in a context related manner in our database. Where does that bring us? We of course have the content of the URLs you shortened, but also the URLs of everyone else using the service, so we can do basically like a google search but instead of having our users typing the search term we do it automatically and instead of doing it on all the pages in the world, we do it on our database and return the URLs that are the most relevant to you (Or so we think). For instance, when you go to your profile page, we randomly select 10 of your previously shortened URLs, crawl through the community URLs and identify the contexts and other people’s URLs that may be interesting you using the URLs you previously shortened.

This could be seen a bit like Amazon’s “recommendation” feature. Even though I know many people don’t like this exact feature of Amazon I still believe, and so do they, that it is still working for many many others :).

Currently on my own profile on short.ie I have links to articles/URLs related to PHP, Politics, Conferences, etc.  As the system grows the content becomes more diversified and the links more relevant. And hey, if we don’t find anything that you may randomly be interested in, we don’t bother showing you a box saying we didn’t find anything, we just show you your profile page without that box

Here’s a bit more details about the recommendation engine:

As many other recommendation engines as such as Amazon, iLike, Collarity, etc. Our recommendation engine makes use of some correlation algorithm (More precisely the Pearson product-moment correlation algorithm). To explain quickly because this is not a whitepaper on statistical analysis algorithms, I will simply say that we firstly take an URL, analyze it’s context and calculate the average of the values creating a median with the values. Then we take the other context urls, analyze them, and calculate the ones with the most points near the median line. The 10 ones with most points are kept and displayed to the user. Badly worded? See the graph below, it should help understanding

Of course this is only an example, but I’ll explain how we use this. The red line is the median of an URL’s point of associations. Then we analyze the other URLs that have contexts and give points depending on wether they match certain relations with the original URL’s median. The URLs that acquire the most points close the the median line are kept/chosen and recommended to the users. Of course the more data you have, the more analysis you can do and this is something we are currently refining and improving daily. We are adding rules to the analysis and making sure that the URLs that we are “recommending” to our users are better and better.

For those interested, here’s a rough “mathematical-representation” of the algorithm.

where

are the standard score, population mean, and population standard deviation(calculated using n in the denominator).

The result obtained is equivalent to dividing the covariance between the two variables by the product of their standard deviations (Note that this is the general equation and the broad idea behind the recommendation system).
That should cover most of the idea and theory the recommendation system. Other things that I have to mention are upcoming features as such as the “public links” (A list of public links, a public timeline ala twitter that will show everyone’s most active public links - and not the links per user, but the PUBLIC TIMELINE ), “top URLs” (A list of the most clicked URLs), top users, referer URLs, etc.

You can leave a response, or trackback from your own site.

Comments (4 Responses)