Lecture Presentation on Linked Data, SPARQL and Semantic Augmentation

Lecture linked data cloud & sparql from Dhavalkumar Thakker

Lecture semantic augmentation from Dhavalkumar Thakker

Help Documentation

Help documentation tool

For some recent work, I was looking for a help documentation tool and was drawn to this  tool called Dr Explain by this helpful & detailed review. It is an excellent tool that generates slick looking help files with:
- on-line help in the form of a high-quality, coordinated set of HTML pages, which can be hosted on a website, complete with table of context, index, and full text search capabilities built in. There is also option of generating PDF and RTF files:
-a PDF file containing all of the HTML help content in sequence, with both bookmarks (tree-structure like a table of contents in a window usually on the left side of a displayed PDF document), and a table of contents with links to the specified pages, and
-an RTF file, with a structure much the same as the PDF output, providing the ability to use it directly or import it into other tools for further processing.

Aggregating Digital Traces in Semantic-enriched data Cloud

Latest work: Presentation at Linked Learning(Lile2012) Workshop at WWW 2012Integrating digital traces into a semantic enriched data Cloud For Informal Learning

View more PowerPoint from Dhaval Thakker

Lesson Learned : Always use "_" for Ontology Concepts

While Defining ontological concepts - I am finding out building an app it is always better to use "_" pattern. For example, "Anatomical_Structure" instead of often  used "AnatomicalStructure". It is so much convenient for Text Mining Systems and for building interfaces.


More later...

Sentiment Analysis on Social Media

This is an interesting article on the topic. Here is one of the quotes that I think is blunt but spot on.

"assuming that you manage to get cleaner data from twitter..


then you have to figure out what people actually are saying “in whatever language it is they’re writing in. 


"We’ve put years and years of work to understand grammar, capitalization, and crap like that, and there ain’t none.”

Ontology 101 using Protege

As part of my work, I have created a basic tutorial on using Protege for creating an ontology that I am happy to share here. 

Linked Data Cloud for the Local Goverments

I was listening to the latest episode in the row between my home council (http://www.nottinghamcity.gov.uk/index.aspx?articleid=1) and the Communities Secretary Eric Pickles(http://en.wikipedia.org/wiki/Eric_Pickles) where Eric Pickles called the council Deputy whose name is Graham Chapman (remember Monty Python ?) "very naugthy boy".  The reason for such comments from Mr Pickles is because the Communities Secretary has urged all councils to publish the expenditure details as part of a "government revolution in transparency", although they will not be forced to do so by law. Nottingham City Council is I think one of the very few who hasn't obliged. Here is what Mr Chapman has to say: "It costs virtually nothing - a couple of thousand pounds - to put it online. It costs another couple of thousand pounds to keep it going online. It's likely to cost about £100,000 to service it.


The jokes and politics (or politics as joke!!) aside, here is a great chance for local governments to take part in the Open Government Data movement and specifically expose the data using Linked Data Cloud way. It might have similar costs to put in online but the service costs can be significantly reduced and can allow the developers in the wild to generate interesting statistics and applications out of it.  

JAPE grammar - performance bottleneck

({Token})* pattern in the JAPE grammar can severely hit performance especially while performing Text Mining on large piece of text.  As advised in the GATE user guide, if the if you can predict that you won’t need to recognise a string of Tokens longer than x. Then it is possible to utilize 
({Token})[0,x]


However when it is not possible to predict it, are there any workaround? I am trying to figure out one.
If anyone else can suggest one?

Semantic Web - Story so far

As the name of this blog suggests, I am interested in realizing Semantic Web vision so that it becomes a norm technology and any web developer can utilize it in development tasks and make sense of it.

Any technology takes off when there are good tools and standards around it and when big players realize technology's potential and start using it or pushing for it. Clearly, when I started doing the research in semantic web technologies (around 2004) that was not the case.

I still remember getting confused and being too "theoritical" about "Description Logic(DL)", "Full" and "Lite" versions of OWL and which one was the best one. To add to the confusion, SWRL (Semantic Web Rule Language) was announced stating that OWL has not got enough expressivity(!).

Also when come to tools there were Jena and Pellet reasoners that were two of the main options where one had to rely on them to do wonderful (semantic) things at the Schema level. but when it came to doing the same on a serious scale (on data/instances) they all struggled big time. We looked at using MySQL as backend and these schema reasoners to do heavy lifting on the reasoning front and so on. In terms of query language there was none and we did all the application logic using custom APIs. And yes, there was no sight of Linked Data Cloud.....

Roll on year 2010 and ..things have changed...big time..

We have query languages, tools, reasoners, triple stores and above all linked data cloud. I have been around the Semantic Web community long enough to identify the key players (companies/personnel) that have made this difference and in this blog I would like to acknowledge them.

1. Original Semantic Web team of Tim Berners Lee and Jim Handler.

Tim Berners Lee for believing when we all thought Semantic web might not work and will be another AI failure. And of course for his His work at the W3C.

James Handler - in addition to his continued work on Semantic Web, for coming up with gems such as the definition of Semantics/Linked Data Cloud that is most effective. In a conversation he defined necessasity of semantics as "My document can point at your document on the Web, but my database can't point at something in your database without writing special purpose code. The Semantic Web aims at fixing that."

2. I am surprised that people in the community does not flag this up but we were not able to sell "Semantic Web" mainly due to the fact that performance of semantic tools was so "poor". There are players in the market that have addressed that and we have a great deal to thank them. From my own experience and market observation, I consider (in no particular order of preference)

OntoText's BigOWLIM

OpenLink's Virtuoso

Talis Platform

the main players in this.

3. DBPedia & Linked Data Cloud

The world was waiting for a large scale semantic datasets and it arrived in terms of "Linked Data Cloud" and "DBPedia" where they had shown for the first time that millions of RDF Triples running using OWL/SKOS ontologies can be produced and managed. It not only demonstrated the practical working side of Semantic Web but also introduced a new architecture for data sharing/integration. The architecture was practical, clearly defined and easy to understand something one sarcastically can say was not associated with "Semantic Web technologies/specs" before.

For this, whole of the linked data community and DBPedia team deserves kudos.

4. I believe that any organisation that needs to take advantage of semantic technologies has to look at Text Mining at some point too because superimposing a new semantic structure is going to be cost prohibitive and text mining can be the solution in many cases. In this regards, toolkits such as GATE, UIMA or services such as OpenCalais that have incorporated support for URIs and Semantics deserves special mention.

5. As mentioned earlier, coming up with a SQL-like query language was a smart move to encourage adoption of RDF technologies into the hands of seasoned Web developers. It all made sense for them once SPARQL was out there.

6. OWL/RDF/SKOS

I know OWL has its ills and people have their reservation about it. I believe that is more to do with its presentation and efforts to portray (at least that is how it might appear to a novice of the technology) it as a specialized field. I believe anyone who have done "Object oriennted programming" and come from OOP world can easliy make sense of OWL.

RDF is already here and is going to stay.

I believe SKOS has a great potential in many E-Commerce taxonomical, and classification applications.

7. Google Refine and similar efforts.

We still have lots of data in CSVs, XLS files (see data.gov.uk) and as we do not expect everybody to use Linked Data, we need to deal with the fact that data from textual format has to be converted into RDF (or other variations) and Google has provided a solid tool for this. XLWrap is also in the same genre.

8. BBC & other case studies

BBC team to come up with the Football World cup website and talk about the technologies and processes in open which is definitely a big booster for semantic web as a product.

Case study of Best Buy, Google acquisition of Freebase are also significant events for semantic web uptake.

More to come as I would like to make it a list of 10 - probably your inputs can complete the list so comment on..

Text Mining and Linked Data Cloud

The text mining technologies and tools have been around for past decade. The way most of the text mining engines work, they require a good set of bootstrapping entities in order to perform well (w.r.t. Precision and recall). These bootstrapping entities are called Gazetteers/ Authority Files/ Lists etc. in different tools.

With the emergence of Linked data cloud and its open datasets there is a great opportunity to utilize Text Mining to achieve even better results where entities in these datasets can be utilized in bootstrapping.

Here is my take on utilizing Linked Data cloud with information extraction system GATE.
Presentation at GATE course in May, 2010.

Benchmarking

This blog relates to the files mentioned in our paper:

A Pragmatic Approach to Semantic Repositories Benchmarking
Dhavalkumar Thakker, Taha Osman, Shakti Gohil, Phil Lakin

Please find the PA Dataset queries as a PDF file here.
Please find the UOBM Dataset query results here.


Please note that UOBM is the work done by these authors.

Update: The query execution timing results mentioned in the paper needs to be revised with respect to new findings about how BigOWLIM and Sesame query execution mechanism works. We will publish the updated results and findings soon.

SPARQL - extension

I know that there is a working group that is looking for extending the current SPARQL specification/protocol. There are also some discussions outside the working group that outlines requirements for such an extension.

As a contribution to these efforts, in this post I want to start outlining things ( queries ) that are not possible to write using SPARQL in its current state. Requirement of such query syntax and support stems from demands of practical Semantic Web development work. Here are some examples. I will appreciate your comments if I am incorrectly assuming that these sort of queries are not possible to write.

1. A query to verify the cardinality of property values.

For example, in DBPedia, a Band (<http://dbpedia.org/ontology/Band>) has property <http://dbpedia.org/ontology/genre> with no cardinality restrictions (i.e. Band can have any number of genres).

Using sparql one can not extract Bands which has more than x number of genres?

The following SPARQL query is not possible to write and execute:

Select ?band

where

{

?band rdf:type <http://dbpedia.org/ontology/Band> .

?band <http://dbpedia.org/ontology/genre> ?genre .

FILTER (COUNT(?genre) > 3).

}

2. Support for negation. It is well known that SPARQL does not support negation. It is obvious that negation is an important feature to have, I will demonstrate the usefulness of negation using one scenario.

In DBPedia ontology, the classification hierarchy for "Organisation" is as follows:

Organisation(http://dbpedia.org/ontology/Organisation) ->

dbpedia-owl:Band dbpedia-owl:RadioStation dbpedia-owl:Non-ProfitOrganisation dbpedia-owl:Legislature dbpedia-owl:Company dbpedia-owl:EducationalInstitution dbpedia-owl:SportsTeam dbpedia-owl:MilitaryUnit dbpedia-owl:Broadcast dbpedia-owl:TradeUnion

If you are mapping your ontology to DBPedia's and if you have a slightly different hierarchy where "SportsTeam" is not a subclass of "Organisation" then

in order to retrieve all the instances of Organisation class that are not "SportsTeam" you will need to use negation (ALL(Organisation)-ALL(SportsTeam)).

The alternative is cumbersome solution where the query will involve retrieving instances of "Organisation"subclasses except SportsTeam and then merging them. For example,

Regex support in CONSTRUCT queries:

One of the purposes of CONSTRUCT queries is to map ontologies and datasets. It will be quite useful to be able to specify regex based URI patterns as part of the CONSTRUCT queries. Particularly applicable where one wants to use the unique & human readbale identifiers from DBPedia. For example, it will be useful to have something like:

CONSTRUCT
{
<http://myurischeme/resource/"unique id from DBPedia URI"> rdf:type myOntology:myType.
}
WHERE
{
<http://dbpedia.org/resource/ABC> rdf:type <http://dbpedia.org/ontology/Person>
}

JAPE grammar tutorial

We are pleased to share the JAPE grammar tutorial we have designed here at Press Association Images, UK. The tutorial can be accessed by clicking here. It is a Zip file including tutorial in PDF formant and few support files.

Please send your comments through this blog...

Converting from one version of semantic data to another (.N3, N-Triples, RDF/XML, OWL)

I have found a very useful little tool that allows converting data (schema/instaces) to and from .N3, N-Triples, RDF/XML and OWL. This utility is particularly handy when your RDF stores accepts schemas in particular format only (in majority of cases N3/N-Triples). Here is the link to the tool and syntax which allows you to do so ( It took me sometime to find the exact syntax so I hope it will save some of your time).

http://www.w3.org/2000/10/swap/doc/cwm.html
&
http://infomesh.net/2001/cwm/

syntax to convert from RDF/OWL to N3.

python cwm.py --rdf file-to-be-converted.owl --n3 > file-as-output.n3


or even better....
you can use Protege :)
Open .owl file in protege and go to File -> SaveAs and save it as .N3 or turtle. So easy, isnt it?

Ontology mapping using SPARQL (Triples extraction from DBPedia)

I have been using SPARQL CONSTRUCT for ontology mapping. One of the problems, I have encountered during my work is, how do I get around BNode in the output graph, as BNode are ugly, not shareable (if thats right word!) and you can say not at all useful here. What you want in the target graph is a clear URI, as you might want to share this new URI with other graphs you extract from different source ontologies.

I stumbled on this post, which have similar conclusion. Somewhere in the post the author mentions that,

"it appears to be impossible to create new URIs for the resources in the target ontology - only bnodes can be created on the fly."

and they had to use some post-processing script to manage bnodes. I am still working on finding a workaround, if I can not then I will have to conclude the same.

Based on my findings, I will update this post soon....

OWL benchmarks

There is a great deal of confusion about the tools to realize semantic web applications. This confusion stems from the fact that there are a large number of tools available and the benchmarks that compare these tools are quite diverse.

For the purpose of the clarity, I will summarize these tools and benchmarks here. I intend to comment on them as much as I can during next few weeks.

(As of January 2008)

OWL reasoners: ( for reasoning, inferencing)

1. Pellet - free/open source with some restriction on commercial use - Supports OWL DL, DL Safe SWRL rules.
   Java based
   
2. RacerPro - Educational free evaluation licence, commercial paid licence - Supports OWL DL
   Java based
   
3. KAON2 - Free for educational and non-academic use, different licence for commercial use. Supports OWL-DL and SWRL with F-Logic ontologies
   Java based
   
4. FaCT++ - seems free,open source
   C++ based
   
5. OWL API - free, open source. This is actually OWL API with reasoner interface to FaCT++ and Pellet.
   Java based
6. OWLIM - free version with limitations, commercial licence is with all features
   Supports RDFS and most of OWL-Lite. More importantly contains a semantic repository over Sesame.
   Java based

RDF Stores (for storing RDF triples)

1. Virtuso from OpenLink
   Getting started link is here
2. AllegroGraph from Franz Inc.
   Java/Lisp based
   Java learning centre is here
3. Mulgara (previously Kowari)
4. SwiftOWLIM and BigOWLIM from ontotext
   SwiftOWLIM is an open-source library, under LGPL
   BigOWLIM is available under an RDBMS-like commercial licence on a per-server-CPU basis
5. Sesame RDF store from Aduna
6. Oracle 11g has RDF store support
7. Scalable Ontology Repository (SOR) [Previously Minerva] from IBM. It supports DLP (Description Logic Program), a subset of OWL DL.
   Part of the IBM Integrated Ontology Development Toolkit.
8. D2RServer is a a tool for serving Linked Data views on relational databases.D2R server relies on a declarative mapping between the schemata of the database and the target RDF terms. Based on this mapping, D2R Server serves a Linked Data view on your database and provides a SPARQL endpoint for the database.

Bechmarks to evaluate these reasoners/storage systems:

1. Lehigh University Benchmark (LUBM)
   
   Y. Guo, Z. Pan, and J. Heflin. LUBM: A Benchmark for OWL Knowledge Base Systems. Journal of Web Semantics 3(2), 2005, pp158-182.
   
   One of the early works on the subject, however now seems outdated. Good reference point for any future comparisons.
   
   Compares the RDF stores.
   timeline: 2004/2005
   
   
2. Berlin bechmark
   from Freie Universität, Berlin
   Benchmark for comparing the performance of storage systems that expose SPARQL endpoints ( does not compare the reasoning capacity)
   timeline: 2008
   
   
3. University Ontology Benchmark (UOBM) from IBM
   
   Li Ma, Yang Yang, Zhaoming Qiu, GuoTong Xie, Yue Pan, Shengping Liu. Towards a Complete OWL Ontology Benchmark. In Proceedings of ESWC'2006. pp.125~139
   
   Compares the RDF stores along with its reasoning capability (OWL DL and OWL Lite).
   timeline:2006
   
   
4. Survey paper : What's wrong with OWL Benchmarks?
   Finds (rightly) fault with above (1 and 3) benchmarks and outlines the requirements that if implemented in a benchmark shall ideally cover real world scenarios.
   
   
   This list is no way near exhaustive, so I plan to extend it whenever I can. Please drop comments if you have any...

Ta..

First post

Welcome to my Blog!