Science for SEO

Well low and behold (you know me by now) I found a paper which addresses this issue in a very intelligent way and answers quite a few questions.  It's called "What is an analogue for the semantic web and why is having one important?" by mc schraefel from the university of Southampton (which regularly produces cool papers).

First off, an "analogue" in this sense is something that has a direct resemblance to something else.  For example print (books and things) is analogous to the web because a web page is like a page from a book or newspaper, manual and so on.  The central topic of this paper is about finding something analogous to help people understand the semantic web so it's not so foreign and scary or whatever.

The author says that the web has been represented by Pages + Links and proposes that the semantic web be represented by Notebook + Memex.

The Memex was invented by Vannevar Bush in 1945.  It's the concept of an online library, an interconnected knowledge-base.

He points out that the semantic web offers very powerful way to interact with information, and to build new interactions for that information.   He does, as I do, believe that the entire issue concerning the semantic web and its acceptance is due to the research community not communicating it properly.

"It is important to note that the motivation for this question of analogue is not a marketing/packaging question to help sell the Semantic Web, but is simply a matter of fundamental importance in any research space: it is critical to have both a shared and sharable understanding of a (potentially new) paradigm. If we do not have such a shared understanding, we cannot interrogate the paradigm for either its technical or, perhaps especially, its social goals."

He notes that all web 2.0 stuff has been based on highly familiar models, as RSS for example is still text and the idea of tag clouds and tagging is still displayed just like a catalogue.  The understanding of the web when it was 5 years old is very different to the understanding of the semantic web which has just turned 5.  He also very nicely points out that the wikipedia entry for the semantic web is a bit rubbish: "All that description tells anyone about the semantic Web is that it is for Machines."   The emphasis as far as semantic web researchers are concerned is the end user.  It is all about creating powerful links in information so that question-answering is made possible and a whole host of other knowledge discovery methods.  

"But how do we describe this potential? For a community steeped in rich link models, Hypertext is an obvious conceptualization. But beyond this community, Hypertext equals “a page with links” – it equals the current Web, not the rich possibility of what we might call Real Hypertext, which was modeled in Note Cards and Microcosm."

The hyperlink in the semantic web can be thought of as "meaning" which is one of the hard to grasp concepts.  "That is, the way meaning is communicated that is not via the explicit prose page or catalogue page, but is via the exposure of the ways in which data is associated, and can be discovered, by direct semantic association, for the reader/interactor/explorer to make meaning."

The author says that the notepad is a good way of introducing the semantic web because it is a page but is unstructured,  Calendars and things can be shared, and things like Twitter allow users to post snippets.  "One item can act as a way of redefining another".  Google docs and various snippet keepers on the web (like the now defunct Google notebook) mean that data is generated and shared and linked in too.  The memex is good too because it is designed to retrieve data and not "denature it" by this we mean that it isn't taken out of context.  

The semantic web languages like RDF mean that everything can be connected and retrieved intelligently and effectively.  It is "automatic structure extraction" which means that you can look at all of your information in the context it was created or saved in.  The data can also be associated to other relevant information too.  For example you could find other people talking about the same topic, or working on the same kind of project, events you might be interested in...

The author observes that the notepad model has limitations like the structure and also "viewing page 6 next to page 36" isn't easy.  He suggests the note card model which is a stack of cards with ideas, people, data about stuff written on them and interlinked to external data.  "The relevance of the note card model to the concept of the Semantic Web as personal work space with associated public data is in the integration of personal ideas with external sources: the idea cards are backed up with/informed by the quotations from external sources."

He concludes asking if we are ready for a system to support such creativity.  This means that computers are no longer simply used for productivity.  He does believe that we are.  I also believe that we are.  More importantly I believe that the semantic web is a foundation for far more intelligent systems.  We need to experiment with this model and develop it properly in order to advance.

Obviously this kind of system can't be implemented by humans completely and needs to be automated, which it is really.  There are all sorts of programs out there that will create stuff for you.

If you don't embrace the semantic web it is likely that you will be left behind because it is a necessary step in web evolution.  It will become widespread and it is important to be prepped for it.  It should have your support because it is an ingenious and very powerful way of dealing with the ever growing mass of data available.

On the Cusp: "A global review of the semantic web industry" by David Provost is a great 38 pager on the semantic web industry at the moment.  It's good because I don't see many buzz words in there or very technical terms, it's high level as it should be for managers, vendors, strategists and customers.

The report is all about the semantic web surprise surprise! He focuses on what companies involved in the semantic web do, and not so much "how they do it".  Twine, Primal Fusion  and the Calais initiative have of course been quite important in this area.  Bottom line, people are investing in semantic web technology.

I have a great love of Twine.  I can spend hours and hours reading content on there and creating my own information groups and discovering new things.  If you're short of something to write about, visit twine.  (Actually I just spent 20mins on there when I only went to copy their url!).

The Calais initiative I am also a big fan of but in a different way.  They provide tools to help connect information together - it's like lego.

Primal Fusion are in Alpha and I haven't had access as yet, but they look like they have something interesting to share.  A way to explore and organise your thoughts.

Back to David Provost:

"Intriguing possibilities are emerging, such as the role of “linked data”, Social Network Analysis and how the Semantic Web may aid this practice, and how the convergence of Natural Language Processing, Semantically enabled search, and the traditional publishing industry will play out. Time will tell, but the potential effects could be substantial".

It's interesting that he says that now deployers have to establish credibility and show that they are more than "2 guys in a garage" - times are changing.

He says that the SW is a global industry, and that vendors are thriving.  Some companies have already started using SW technology in risk management, knowledge management and other areas.

He mentions Franz'z Allegrograph which is a semantic database which I really like.  You can use a free version and they also do the RacerPro reasoner.  It can be used for social network analysis.

NLP is "emerging as a force for taming world wide content", basically quality content and wild content which is unknown.  So basically this field continues to grow and evolve as we are well aware.

Linked data is going to have valuable and has super important uses.  This is one of the primary aspects, we touched on this with FOAF earlier.

Marketing, technical and solution patterns will have a greater role in selling semantic web solutions.

The author says that we're going to move away from the very low level concepts and terminology used by researchers and move on to a much higher-level of discussion.

 

So if I have this right, the semantic web can now be marketed so we need to use higher level language and focus on what it can do for companies.  Fair nuff.

Tad over at the CogBlog has some interesting questions for you and has some interesting ideas.  Swing by and take a look.

This cool little tool is created by a team consisting of NLPers, AI aces, and the CEO is Neil Roseman (former VP of technology at Amazon).  The idea is to allow content to network and help you discover new connections in your semantic ride.

"Evri's technology automates connections between Web content by applying a more human-like understanding of the words on the page. We think that there is a big opportunity to help website publishers better engage their readers and help readers discover compelling content in a new and addictive way."

It doesn't use keywords, popularity or anything other than each element available (place, person, article...) and how to connects to others.  Evri grabs content from highly rated resources and is constantly working to increase it's knowledge base and building a nice data graph of the web at the same time.  

I like it :)

The fact that you never actually type a query to get information is a trend I see continuing waaaay into the future.  There will be a time when there is no need to visit a search engine as all of your information will be available to you in whichever environment you happen to be in (word doc, excel, programming IDE, game...).  Personalisation supports these applications, imagine how much a system can learn from you in 5 years of working with you.    

So we'll be needing a fair few more of you to join the ranks in A.I please, and bring your HCI friends too.  

 

It refers to meaning in language (or code or anything else).  It used syntax and pragmatics as well as contextual information to provide the meaning of the text or even audio stream if you want to use that.  It's not just about finding similarities or context between 2 words but rather taking the entire text or query to establish meaning.  

It's a common framework allowing information to be shared and reused.  Information is stored in machine readable formats.  

"The Semantic Web is an extension of the current web in which information is given well-defined meaning, better enabling computers and people to work in cooperation." - Tim Berners-Lee

That's probably the best definition, seeing as he invented it.  It's  relevant to semantic search and uses many of the same techniques and also is based on the same idea.  It's not a new version of the web, but rather an extension.  There are a lot of conferences about this worldwide, such as "The semantic web technology conference" for example.  

Technologies used obviously include ontologies (which are like big storage boxes full of information on how words and concepts link to each other).  These are built in OWL mostly, Natural language processing tools for named entity extraction for example, Data interchage formats (like RDF/XML or turtle for example), schemas like the RDF Schema, XML to provide syntax for content structure and SPARQL which is a web query language for semantic resources.   

Google uses PageRank to identify relevancy, whereas semantic search engines use meaning to return highly relevant results.  Google returns keyword/keyphrase results, and the semantic solution returns information.  

The data has to be really structured in ontologies just like in the semantic web.  A semantic network is created which links all of the concepts and words together.  It used word sense disambiguation (WSD) in order to decipher what a word may related to.  WordNet, which you can download for free, is a machine readable dictionary that a lot of scientists have used for this task, although it's far from foolproof.  Here is a very comprehensive list of which semantic search engines use what kind of procedure, in pretty plain English.

Google does respond to natural language queries, such as "Where was Marilyn Monroe born?".  Hakia doesn't understand the query and tells me what Marilyn's real name was.  Powerset (only searches Wikipedia and Freebase) comes up with the goods, "LA".  Working in a "closed domain", Powerset has an easier job than Hakia who searches the whole web, just like Google.  Google however delivered where Hakia didn't this time round.

Then I tried in all 3 "Is chili bad for you?" - Hakia came up with books reviews for a book called "Bad chili", Google came up with a forum article with that exact question in it, and Powerset delivered an in depth article on the effects of chili on humans.  The following results for Powerset are all off though.  Hakia results continue with the book, but Google gives me loads of results all about the effect of chili on the body.

Have a go yourself and see what happens.  

This little test definitely shows that Google can come up with the goods, whereas the semantic engines struggle.  More work needed there.  I would be very surprised if Google were shunning semantic web technology and natural language queries.  I would leave that open for discussion actually.

The future?  Natural language queries, and natural language generation for a straight answer to a question, and a summary of all of the most relevant resources in one text, and the option to read the individual documents.  That's not an easy feat!