Posts Tagged ‘linked’

MonetDB and OpenJena

April 6, 2012 1 comment

MonetDB has been updated recently with a Dec 2011-SP2 release. Having previously tried to integrate it with OpenJena and failed because of the use of multiple inner joins, I was happy to find that the update fixed those problems and allows all the integration/unit-tests to pass.

This means of course that Im going to now have to create a patch to Jena (see Jira issue[1]), and when thats done, you can follow the instructions below to test it out – literally run the unit tests. I have been using Ubuntu 11.10 amd64 for this so the notes below reflect this:

1) Download latest MonetDB and JDBC driver

2) Install as per instructions (default username:monetdb with password:monetdb)

3) In your home dir create a my-farm directory

4) Create an "" file to house your local settings for PATH etc

export JAVA_HOME=/usr/lib/jvm/java-6-sun
#point this to whereever you have SDB installed
export PATH=$SDBROOT/bin:$PATH
#point this to whereever you have downloaded the MonetDB JDBC driver
export SDB_JDBC=~/Downloads/monetdb/jdbcclient.jar

5) Create a "monet_h.ttl" assembly file to define a layout2/hash repository

@prefix sdb:     <> .
@prefix rdfs:     <> .
@prefix rdf:     <> .
@prefix ja:      <> .

# MonetDB

<#store> rdf:type sdb:Store ;
sdb:layout     "layout2/hash" ;
sdb:connection <#conn> ;

<#conn> rdf:type sdb:SDBConnection ;
sdb:sdbType       "MonetDB" ;    # Needed for JDBC URL
sdb:sdbHost       "localhost" ;
sdb:sdbName       "TEST2H" ;
sdb:driver        "nl.cwi.monetdb.jdbc.MonetDriver" ;
sdb:sdbUser        "monetdb" ;
sdb:sdbPassword        "monetdb" ;
sdb:jdbcURL    "jdbc:monetdb://localhost:50000/TEST2H";

6) create a script – ""– to drop,create and initialise the repo – this needs to be used each time you run the sdbtest suite. It will make use of the and the monet_h.ttl

monetdb stop TEST2H
monetdb destroy TEST2H
monetdb create TEST2H
monetdb release TEST2H
. ./
bin/sdbconfig --sdb monet_h.ttl --create

7) Run the script

8) Check things went ok with

i) mclient -u monetdb -d TEST2H.

ii) \D

You should see a dump of the schema. There should be among other things a prefixes table.

9) Now for the unit tests :

Create a monetdb-hash.ttl file that Jena can use to connect with

@prefix sdb:     <> .
@prefix rdfs:     <> .
@prefix rdf:     <> .
@prefix ja:      <> .

[] rdf:type sdb:Store ;
sdb:layout     "layout2" ;
sdb:connection _:c ;

_:c rdf:type sdb:SDBConnection ;
sdb:sdbType       "MonetDB" ;    # Needed for JDBC URL
sdb:sdbHost       "localhost" ;
sdb:sdbName       "TEST2H" ;
sdb:driver        "nl.cwi.monetdb.jdbc.MonetDriver" ;
sdb:sdbUser        "monetdb" ;
sdb:sdbPassword        "monetdb" ;
sdb:jdbcURL    "jdbc:monetdb://localhost:50000/TEST2H?debug=true&logfile=monet.debug.log";

10) If in Eclipse, with the SDB source, create a run configuration for sdbtest.

#Main class : sdb.sdbtest
#Arguments: --sdb monetdb-hash.ttl ./testing/manifest-sdb.ttl

11) Run the test suite – all tests should pass.

12) Next : Load some RDF and test performance !……


DERI (LATC) launch

June 18, 2011 Comments off

Some of the DERI people (and others) involved in LATC have launched to counter the lack of rdfs in – the Microsoft/Google/Yahoo attempt to kickstart some RDFa publishing so their search engines can try and improve result relevancy. Some of the items in are quite simple, but thats probably a good thing : a large term set or number of properties is going to look daunting to anyone interested or someone starting out for the first time – and indeed this is the reason cited that it is not RDF (its microdata). And while I agree with Michael Bergman that it is more than likely another step towards structured/linked/common/open data, adopters urgently need a combination of

  1. Tools (or better still no tools, just an unobtrusive natural way to author microdata or rdfa) and
  2. a Reason to do it – payback
  3. Support in search UIs to specify vocabulary items

I’d like a wordpress plugin for instance, but then I’d need to host an instance myself or find a hoster that allows plugins because doesnt allow it. I’d also like to think that if I placed some RDFa in my blog that it would get higher a ranking in Search results (it should) but this blog is pretty specialised anyway and its not commercially oriented so Im happy enough with keyword based results anyway.

So, I’m not going to be doing it too soon, and thats the problem really. Or is it ? This post isn’t data really, but it does have links and it does talk about concepts, people, technology problems. If I could mark them up with tags and attributes that define what I am talking about then it would mean that I could tell those search engines and crawlers what I am talking about rather than hoping they can work it out from the title, the links I have chosen, then feedback comments and so on. Then people looking for these particular topics could find or stumble upon this post more easily. So, while there is some data here, arguably I don’t see it that way, and even if I think there might be a good Reason to do it, it’s too hard without the Tools

So I wonder finally, if I was to mark up one of these people mentioned in this post with name,address, affiliation,organisation and so on, would the search engine UIs allow me to use this vocabulary directly – I want to find articles about DERI say, would the search drop down prompt me with itemprop="EducationalOrganization" – so that I’d then only get results that have been marked up with this microdata property and not with things that are about the Deri vineyard in wales, punto deri, courtney deri and so on ?

Sindice kinda does this, couldn’t the Goog do it too ??? Or indicate which results are microdata’d, or allow a keyword predicate (like site: say), or allow the results to be filtered (like Search Tools in the left column). The point for me is that is only half or less than half the story – the search engines need to Support the initiative by making it available at query time, and to allow their results to manipulated in terms of microdata/rdfs too. Then I might be more tempted to markup my posts in microdata,rdfs,microformat or whatever, and I might create some extensions to the schemas and contribute a bit more, and my post might get more traffic in the long tail, that traffic would be more valuable, my ad revenue might go up (if I had ads for myself !), and the ECB might drop their interest rates. Well, maybe not, but they’re not listening to anythine else, perhaps some structured data might persuade them. It is the future after all.

Cisco go Social Semantic

February 22, 2011 Comments off

An announcement today[1/2]* of a cash injection to DERI from Cisco in respect of developing Cisco Quad[3] with a Semantic angle (SIOC no doubt) and Data Integration/Recommendation/Social-Network-Analysis with the proposed ADVANSSE project.[4,5]


Quad boasts standards like JSR-168, JSR-286, OpenSocial integration, XMPP, CMIS[6], but seems proprietary[7], and perhaps stuck with MS Exchange. Could this be a move to open it up for variegated enterprise, ECM and Web 3.0 [8]?












*Same story different logo

Categories: business, linked, semantic Tags: , , , ,

Aperture Nepomuk queries

February 22, 2011 1 comment

Having crawled an Imap store (ie google mail), I now need to query the results to see whats what, whos who, and how they are connected, if at all.

These are the namespace prefixes used in the queries

Prefix URI

And these are the queries. Note that each message is in its own graph, and references the folder in which it rests – eg <imap://;TYPE=LIST>. This in turn nie:isPartOf another folder, which isnt nie:isPartOf a parent folder.

An Imap store has a username and password etc, but doesnt have an associated email address. A folder may contain messages to the owner with an email address the server accepts, but may also contain messages to other addresses if the cc list contains the owner address.

id Folder Relationship Purpose Query
inbox direct Basic find list of emails, with
sender email address
select distinct ?subject
?from ?address{

?s nmo:from ?o  .
?o nco:fullname ?from .
?o nco:hasEmailAddress ?e .
?e nco:emailAddress
?address .

?s nmo:messageSubject
?subject .

?s a nmo:Email} 

note : with a Jena TDB dataset, use

select distinct ?subject ?from ?address{
graph ?g{
?s nmo:from ?o  .
?o nco:fullname ?from .
?o nco:hasEmailAddress ?e .
?e nco:emailAddress
?address .
?s nmo:messageSubject
?subject .
?s a nmo:Email

inbox direct Find emails, distinguish
replies(and what replied to), and CC addresses
select distinct ?s ?subject
?r ?to ?refid ?from ?address{

?s nmo:from ?o  .
?s nmo:messageId ?sid .
?o nco:fullname ?from .
?o nco:hasEmailAddress ?e .
?e nco:emailAddress
?address .

?s nmo:messageSubject
?subject .

?s a nmo:Email
optional {
?s nmo:inReplyTo ?r .
?r nmo:messageId ?mid .
optional {
?s nmo:to ?toid .
?toid nco:fullname ?to .
optional {
?s nmo:cc ?ccid .
?ccid nco:fullname ?ccto .
optional {
?s nmo:references ?refid .}} 

order by ?subject

Note : nco:fullname optional as you may not know the email addressee’s name
Note : As with the basic query about, where using a Jean Dataset, you need a graph selector in the where clause eg

select * { graph ?g {?s ?p ?o}}
inbox direct most messages direct to you
select (count(?from) as ?count) ?from ?address{
graph ?g{
?s nmo:from ?o  .
?o nco:fullname ?from .
?o nco:hasEmailAddress ?e .
?e nco:emailAddress ?address .
?s a nmo:Email

group by ?from ?address
order by desc(?count)
inbox direct most messages CC to you Not so easy : where you are a CC recipient, its not possible to match on the to: field, or with any metadata on the imap server.
inbox direct fastest replies
inbox direct most replies
inbox contacts and counts by mail
inbox indirect messages to others on CC list
(may not be known to you, but sender knows)
outbox direct recipents (to,cc,bcc)
outbox direct replies
outbox direct most replied to
outbox direct most sent to
outbox direct fastest replied to (by message,
by recipient)
output direct fastest sent to (by message, by

Things get more interesting when more that one mailbox is available for
analyis…but Im going to need Sesame3 or revert to Jena because Sesame2 doesnt do aggregate functions like count. 2 steps forward, 1 step back. So, Jena support in Aperture is minimal and old. It cannot make use of graphs, TDB or SDB, (but the libraries are up to date). It also doesnt support Datasets or Named Graphs in Jena. So, I add ModelSet (the RDF2Go adapter type needed), Dataset and Named graph support, in TDB to begin with. This involves updating the Aperture Jena adapter. Doesn’t seem to be any activity on the Aperture mailing list tho, as I get zero response to a question about updating the Jena support. Is Aperture another nice-but-dead Semantic Web technology ?

Linked Data, OData, GData, DataRSS comparison matrix

February 17, 2011 4 comments

(Update : Its a year since my next-in-line brother died. This post involved a conversation I had with him. I’ve finally, after 5 years, updated it to include an .ODT version of the table below. He’ll probably kick my ass the next time I see him….)

I’m new to OData, having just talked with one of my brothers about it. He’s using it in a large company, but I’m not sure if its an internal tool or for customers. However, having forgotten or never investigated it before because of the lack of Microsoft fanfare, I was struggling to see what the difference between it and Linked Data with RDF is. Much googling and reading [47,48,50] left me with lots of questions, points of view, some pros and cons, and discovery about GData and DataRSS. (I’ve really been sipping the W3C Linked Data Kool Aid too long 🙂 ).

So, I want to create a matrix of criteria that anyone can quickly look at and get salient information about them. (Perhaps I could publish that matrix as linked data sometime…). You’ll understand by now that I haven’t used OData so I’m going on what i read until I install SharePoint somewhere, or whatever else it takes to get a producer running to play with. And with that, I’ve also made the mental jump to Drupal publishing RDFa – can you embed OData in a web page ? Would you want to if you had a CMS where the data was also content ?

I hope to end up with information about LoD, OData, GData, DataRSS (and fix this table’s formatting). Help ! RDF_OData_GData_DataRSS

Criteria RDF…/LinkingOpenData OData GData

Logical Model Graph/EAV.
Technology grounding (esp OWL ) in Description Logic.[12, 13]. “Open
World Assumption” [27]
Graph/EAV. AtomPub
and EDM grounding in entity relationship modelling [11]. “Closed World
Assumption”[28] view (?) but with “OpenTypes” and “Dynamic Properties”
Unclear/Mixed – whatever google logical Model is behind services, but transcoded and exposed as AtomPub/JSON. Data relations and graphs not controllable by API – eg cannot define a link between data elements that doesnt already exist. GData is primarily a client API.
Not mandated, but probably
backed by a triple store and serialised over Http to RDF/XML, Json,
TTL, N3 or other format. RDBMS backing or proxying possible.
not mandated, but probably
backed by existing RDBMS persistence [4 – “Abstract Data Model”], or
more precisely a non-triple store. (I have no evidence to
support this, but the gist of docs and examples suggests it as a
typical use case) and serialised over Http with Atom/JSON
according to Entity Data Model (EDM)[6] and  Conceptual Schema
Definition Language (CSDL)[11]
Google applications and services publishing data in AtomPub/JSON format, with Google Data Namespace[58] elements.
Intent Data syndication
and web level linking : “The goal of
the W3C SWEO Linking Open Data community project is to extend the Web
with a data commons by publishing various open data sets as RDF on the
Web and by setting RDF links between data items from different data
Data publishing
syndication : “There is a vast
amount of data available today and data is now
being collected and stored at a rate never seen before. Much, if
not most, of this data however is locked into specific applications
or formats and difficult to access or to integrate into new
Google cloud data publishing [55] : “The Google Data Protocol provides a secure means for external developers to write new applications that let end users access and update the data stored by many Google products. External developers can use the Google Data Protocol directly, or they can use any of the supported programming languages provided by the client libraries.”
http, content
negotiation, RDF, REST-GET. Sparql 1.1 for update
http, content
negotiation, AtomPub/JSON, REST-GET/PUT/POST/DELETE [9]
Openness/Extensibility Any and all,
create your own ontology/namespace/URIs with RDFS/OWL/SKOS/…, large
opensource tooling & community, multiple serialisation RDF/XML,
Any and all (with
a “legacy” Microsoft base), while reuse Microsoft classes and types,
namespaces (EDM)[6] with Atom/JSON serialisation. Large microsoft
tooling and integration with others following.[7,8]
Google applications and services only.
URI minting,
Create your own
URIs and namespaces following guidelines (“slash vs hash”) [15,16]
Subject, predicate and object URIs must be dereferencible, content
negotiation expected. Separation of concept URI and location URI
Unclear whether
concept URI and Location URI are distinguished in specification –
values can certainly be Location URIs, and IDs can be URIs, but
attribute properties aren’t dereferencible to Location URIs.Well specified URI conventions [21]
Atom namespace.  <link rel=”self” …/> denotes URI of item. ETags also used for versioned updates.  Google Data namespace for content “Kinds”.[59], no dereferencing.
matching, equivalence
External entities
can inherently be directly linked by reference, and equivalence is
possible with owl:sameAs, owl:seeAlso (and other equivalence assertions)
properties link entity elements within a single OData materialisation –
external linkage not possible. Dereferencable attribute properties not
possible but proposed[10].
URIS Not dereferencable, linkage outside of google not possible.
Data Model :
create ontology model of data, concepts and relations. Import and
extend external ontologies. Terminology (T-Box) and Asserted ( A-Box,
vocabulary) both possible with OWL presentation.
EDM defines
creation of entities, types, sets, associations and navigation
properties for data and relations. Primitive types a la XSD types[35].
Seems more akin to capabilities of schema definition than ontology
modelling with OWL. Unclear at this stage whether new entities can be
created, and then reused or existing ones imported and extended.
Reasoning Inferrence or reasoning out of
DL terminology and assertion separation possible. May be handled at
repository level (eg Sesame) or at query time (eg Jena)
service may be able to infer
from derived typing[41]
Declare namespaces
as required when importing public or “well known”
ontologies/vocabularies, creating SPARQL queries, short hand URIs,
create new as required for your own custom classes, instances.
supported in EDM but unclear if possible to create and use namespace,
or if it can be backed with a custome class/property definition
(ontology). $metadata seems to separate logically and physically type
and service metadata from instance data – ie oData doesn’t “eat its own
dog food”.
AtomPub and Google Data namespace only.
Content negotiation Client and server
negotiate content to best determination.[17,18]
Client specifies
or server fails, or default to Atom representation.[19]. Only XML
serialisation for service metadata.[40]. New mime-types introduced.
Use alt query param (accept-header not used)[57]
Query capability Dereferencibility
central principle to linked data, whether in document, local endpoint
or federated. SPARQL [14] query language allows suitably equipped
endpoints to service structured query requests and return serialised
RDF, json, csv, html, …
dereferencible URIs with special $metadata path element allow type
metadata to be retrieved [10]. Running a structured query against an
OData service with something like SPARQL isn’t possible.
Query by author,category,fields.
Derefencable URIs,
well-known/common/upperlevel ontologies/vocabularies.VoID [22,52] can be used to provide extensive metadata for a linked
Service documents
[32] describe types, datasets. Programmatic Mapping to/from RDF
RDF-XML, ttl, n3
well known formats. See also content negotiation.
AtomPub, JSON
outputs. OpenVirtuoso mapping, custom code.
Security, privacy, provenance. No additional
specifications above that supplied in web/http architecture. CORS
becoming popular as access filter method for cross-site syndication
capability at client level. Server side access control. Standards for
Provenance and privacy planned and under development[24]. W3C XG
provenance group[25]
No additional
specifications above that mandated in http/atom/json.[23, 31] CORS use
possible for cross site syndication. Dallas/Azure Datamarket for
“trusted commercial and premium public domain data”.[26]
Http wire protocols, but in addition authentication (OpenID) and authorization are required(OAuth). “ClientLogin” and AuthSub are deprecated. [60]. No provenance handling.
license, sponsorship, governance
W3C Supported
community project [1], after a proposal by TBL[2]. Built up
Architecture of World Wide Web [3]
Microsoft owned
sponsored under “Open Specification Promise”, [4] but brought to W3C
incubator [5]
support, community
w3c docs,
community wikis,forums,blogs,developer groups and libraries.[38] site
with developer docs, and links to articles and videos, mailinglist, msdn
producing, consuming
Many and varied,
open.[1,36,37] et al.
consumers[8], “datamarket”[26], PowerPivot for Excel[49
Other SPARQL update v1.1[42], Semantic
Web Services, [43-46,51-54]
Batch request [20], protocol
versioning [33], Service Operations [30]





























































Categories: linked Tags: , , , , , , , ,

Open Data experiences at the IIEA

February 16, 2011 7 comments

Yesterday attended the IIEA supported talk from Nigel Shadbolt [1,2,3] about TBL[4] and his experiences of creating and dealing with individual data huggers within the UK public service. About 30 people there and a lively Q&A session followed. The IIEA are to be congratulated I think, and I hope many more talks like this can be held.

Getting into it the content of the talk, firstly, topics for persuading those who may not “see the light” on Open Data are :

  • Transparency
  • Accountability
  • Engagement
  • Improve Public Service Efficiency
  • Economic and Social Values
  • Crowd sourcing error correction
  • Follow the 5 Stars approach – baby steps

Some government departments embrace the idea, others were not so welcoming (eg Ordnance Survey [10]). Local government are often those most enthusiastic about open data because it brings people to them with a context – informed residents make better rate payers effectively  (!) – better decisions about everything from budget allocations, police statistics, bicycle accident blackspots to potholes[6] can be made because of informed two-way communications. MySociety[5,7] volunteers are harnessing this new capability. Local councils also benefit because of the visibility they now have into other councils – they can compare how much they paid for grit last winter for instance, and work out if their money was well spent – something they couldn’t do before.

The Open Data project in the UK is far from complete, but progress is being made.  People want to see and use the data even if its not perfect, people can do things with the data government never would or could, what has been already created is not perfect but it keeps getting better. However, top-down “encouragement” is sometimes necessary and (to paraphrase) a change in attitude from “don’t publish until an FoI request comes in” to “publish by default (bar clearly personal data)” creates an environment for success. The current Tory/LibDem coalition government have taken up the mantle now and Nigel Shadbolt is on the board of the new Public Sector Transparency Board.[9]


Here in Ireland I believe we need someone like Nigel and this needs to be an “information tsar” rather than a minister. The landscape and attitude are different here, and even though Nigel emphasised that the UK project was far from complete or perfect,  I think if we in Ireland had even a portion of that has been done in the UK, we’d be living in a different kind of country.

I’d suggest that Democracy thrives on Open Data, and for the citizens in a democracy, information is power.

The slides for the presentation aren’t available on the IIEA site yet, but I expect they will be soon. In the mean time, I found some similar (but more technical) ones[8] from a previous talk elsewhere.

Categories: government, linked Tags: , , , ,

Turkey Brining with UMBEL

December 16, 2010 Comments off

Festive season comes again. Glad I’m not doing it this year.

This post after (Harold) McGee on Food & Cooking1[1]

Immerse meat in 3-5% w/w salt solution (30-60g/kg) for a few hours – and up to 2 days. A 3% solution dissolves part of the protein structure that supports the contracting muscle filaments and a 5.5% solution partly dissolves the filaments themselves. You also get a greater water holding capacity because of the salt-protein interaction. The inward movement of salt and water also adds the flavours you might add to the solution. The meat will increase in weight by 10% or more. It works from outside in, so incomplete soaking will mean that inner muscle wont be effected so much. But because of the extra moisture and filament breakdown, even an over cooked bird can benefit.

1. Harold McGee, McGee on Food & Cooking, pp155-156, Hodder & Stoughton, London, 2004

umbel misses, but mapped in SUMO[2] : dissolves, structure, filament, capacity, weight, soaking, moisture