Archive

Posts Tagged ‘Installations’

Arduino Mega ADK tty device

December 29, 2011 Comments off

Unlike what most of the documentation on the arduino site refers to, the Mega ADK board makes use of /dev/ttyACM0 and not /dev/ttyUSBn[1]. And in order to use it, you need the latest 1.0 version of the arduino dev kit, which has an updated RXTX library to support it, so don’t go updating it, like all the ubuntu install instructions eventually get around to telling you to do. You may also get into a rats nest of uninstalling tty and ftdi drivers.

So

  1. Get a JDK6 or 7 JDK (32 or 64 bit available)[2]
  2. Download the arduino dev kit (32/686 or 64 bit available)[3]
  3. Install all the avr stuff : 'sudo apt-get install avr-libc avrdude binutils-avr gcc-avr'
  4. Install the downloaded jdk then 'sudo update-alternatives --config java' to set the default JVM.[4]
  5.  Untar the arduino download to a dir of your choice (your home dir perhaps, as you dont want to be doing this as root)
  6.  Plug in the arduino board to a free USB port, and then run ./arduino from the install dir
  7.  Set the ADK board from Tools->Board menu
  8.  Set the serial port from Tools->Serial Port– it should already be done. If its not, or it is disabled, something is wrong with your installation. Check previous steps. Cry. Try again.
  9.  From File->Examples->Basic choose Blink.
  10.  Click the right arrow icon to upload to your board. Check pin 13 amber LED is blinking.

[1]http://www.arduino.cc/playground/Learning/Linux – see last para of “Included Dependencies”
[2]http://www.oracle.com/technetwork/java/javase/downloads/java-se-jdk-7-download-432154.html
[3]http://arduino.cc/en/Main/Software
[4]http://www.shinephp.com/install-jdk-7-on-ubuntu/

Java Semantic & Linked Open Data webapps – Part 5.1

January 18, 2011 1 comment

How to Architect ?

Well – what before how  – this is firstly about requirements, and then about treatment

Linked Open Data app

Create a semantic repository for a read only dataset with a sparql endpoint for the linked open data web. Create a web application with Ajax and html (no server side code) that makes use of this data and demonstrates linkage to other datasets. Integrate free text search and query capability. Generate a data driven UI from ontology if possible.

So – a fairly tall order : in summary

  • define ontology
  • extract entites from digital text and transform to rdf defined by ontology
  • create an RDF dataset and host in a repository.
  • provide a sparql endpoint
  • create a URI namespace and resolution capability. ensure persistence and decoupling of possible
  • provide content negotiation for human and machine addressing
  • create a UI with client side code only
  • create a text index for keyword search and possibly faceted search, and integrate into the UI alongside query driven interfaces
  • link to other datasets – geonames, dbpedia, any others meaningful – demonstrate promise and capability of linkage
  • build an ontology driven UI so that a human can navigate data, with appropriate display based on type, and appropriate form to drive exploration

Here’s what we end up

Lewis Topographical Dictionary linked data app - system diagram

  1. UserAgent – a browser navigates to Lewis TDI homepage – http://uoccou.endofinternet.net:8080/resources/sparql – and
  2. the webserver (tomcat in fact) returns html and javascript. This is the “application”.
  3. interactions on the webpage invoke javascript that either makes direct calls to Joseki (6) or makes use or permanent URIs (at purl.org) for subject instances from the ontology
  4. purl.org redirects to dynamic dns which resolves to hosted application – on EC2, or during development to some other server. This means we have permanent URIs with flexible hosting locations, at the expense of some network round trips – YMMV.
  5. dyndns calls EC2 where a 303 filter intersects to resolve to either a sparql (6) call for html, json or rdf. pluggable logic for different URIs and/or accept headers means this can be a select, describe, or construct.
  6. Joseki as a sparql endpoint provides RDF query processing with extensions for freetext search, aggregates, federation, inferencing
  7. TDB provides single semantic repository instance (java, persistent, memory mapped) addressable by joseki. For failover or horizontal scaling with multiple sparql endpoints SDB should probably be used. For vertical scaling at TDB – get a bigger machine ! Consider other repository options where physical partitioning, failover/resilience or concurrent webapp instance access required (ie if youre building a webapp connected to a repository by code rather than a web page that makes use of a sparql endpoint).

Next article will provide similar description or architecture used for the Java web application with code that is directly connected to a repository rather than one that talks to a sparql endpoint.

Installing JDKs & Tomcat on Ubuntu 10.04/10 AMI on EC2

November 8, 2010 Comments off

That title is a mouthful isnt it ? A cloudful.  Only slightly better than “How to go about successfully installing Sun JDK6 on Ubuntu 10.04/10 (Lucid/Maverick) using an Ubuntu EBS image on Amazon EC2”

I am trying to setup an Ubuntu 10.04 image on EC2 as a t1.micro. Amazon are giving away free time for the first year on this size image so its a good way to get to grips with things and learn the ins and outs.

So

1) get an amazon account 🙂

2) sign up for EC2 – means handing over credit card details and read the quick getting started guide

2.1) download the cmdline tools or use the web interface.  There are some things you need this for because the web console doesnt have everything. I used the web interface to set up credentials and so on, had to set up ssh on cygwin, then switched to putty (use puttygen to convert the amazon private key first). Cmdline tools need a pk and a x509 cert, so you need to follow the instructions to create those as well.

3) find an image – I have unsuccessfully tried

  • ami-480df921 – a basic ubuntu i386 10.04 on EBS
  • ami-508c7839 – a basic ubuntu i286 10.10 on EBS

4) read some documentation usage docs about volumes, snapshots, reboots and termination and whatever you can from the library of docs and tutorials on  pricing, sizing, architecture

5) decide what you need installed for your purposes – you will install software on your instance just like its your own – that is to say there is no special amazon repo or methodology to follow

6) go for it – first change the instance so the root volume is preserved on reboot (otherwise you’ll lost it all)

/dev/sda1=vol-ddf910b5:attached:2010-11-15T17:09:44.000Z:true

6.1) I wanted Java, Tomcat, mySQL and my own software

6.1.1) Java & Tomcat – I used sudo termsel – and chose the Java-Tomcat option – what could be easier ? Well, lots actually. The task hangs, and no amount of ping, ssh or retries helps. So, reboot the instance : this seems to work.  Syslog has nothing to reveal. Try again, same thing happens. Turns out its trashed, so you need to use apt-get purge on the openjdk jre packages it attempts to install then do an apt-get update, then try again. It worked for me, but I suspect YMMV.

6.1.2) MySQL – use apt-get and install the MySQL-server package

6.1.3) my software – jarred up an instance of my webapp, WinSCPd to the instance (very slow), and unjarred. Had to install the OpenJDK (not jre) package to get jar of course. Sigh. Added Ubuntu to the tomcat6 group with adduser (not useradd !!!), copied and moved webapps and libs around the place, then fixed up local paths in my config.

Restart tomcat and cross fingers. My JNDI context name (jdbc) isnt being found and an instrumentation agent Im using with -agent on the JAVA_OPTS is failing. (getAllLoadedClasses() is returning null ! ) Could it be that I need the Sun JDK ?…..

6.1.4) Attempt to install Sun JDK. apt-get install sun-jdk6 you might guess – but thats too easy.  Sun JDK6 is now in an archive repository, so you need to add that

sudo add-apt-repository "deb http://archive.canonical.com/ lucid partner"
sudo apt-get update
sudo apt-get install sun-java6-jdk

and then, wait for it, your instance hangs. So, Im about to try to fix that, by following instructions from an Ubuntu launchpad bug report.

i) In theory, first thing to do is start your t.micro instance as m1.small. You need the cmdline tools for this:

ec2-run-instances -k your-pk-file-name -g your-security-group-name -t m1.small ami-image-id

for example

ec2-run-instances -k .ssh/uoc_kp -g uoc-test-1 -t m1.small ami-480df871

ii) connect to the instance and install java. you’ll need to use the same pk in whatever ssh client you are using. the install java with


sudo apt-get update
sudo apt-get install sun-java6-jdk

iii) stop instance

iv) start normally – that is without setting type to m1.small

However, when you do i) you end up with a new instance with a new volume. So how can you install the jdk on the volume you need ?? Ive asked, but got no answer….

So now I try an Amazon Linux image. This however doesnt have Sun JDK available to install, so I’ll try one more time with the openjdk – maybe this version will work

  • launch instance and connect
  • scp tarball of install from ubuntu instance (install public key from ubuntu user in .ssh/authorized_keys on Amazon Linux AMI)/
  • Link /var/lib/tomcat6 to home dir : ln -sf /var/lib/tomcat6 tomcat6
  • install tomcat6, tweak /etc/init.d/tomcat6.conf so that CATALINA_BASE points at /var/lib/tomcat6
  • add ec2-user to tomcat group
  • chmod 775 ~/tomcat6 so that tomcat group can write logs etc
  • mv mysql connector jar to $CATALINA_HOME/lib
  • install mysql : sudo yum install mysql-server.i386 – but how to set up root user ? more pain !
    • well – when you first run /etc/init.d/mysqld it tells you
      /usr/bin/mysqladmin -u root password ‘new-password’
      /usr/bin/mysqladmin -u root -h ip-10-100-10-100 password ‘new-password’
  • run tomcat6 …..
  • same problem – no instrumentation (my app wont run without)

S0, digging deeper, I find eventually that my JAVA_OPTS dont get passed with sudo. So I use -E, and they get as far as /etc/init.d/tomcat6, but no further. Looking at /usr/sbin/tomcat6 which is called by the init script, I see that /etc/tomcat6/tomcat6.conf is pulled in, and env vars can be created here, including JAVA_OPTS (which has the -javaagent JVM param).  And its writable by the tomcat group, so this is the best place to put it, as long as you wrap it in quotes.

So, now, tomcat6 appears to be starting, but I have other problems to solve, but at least theres progress. Snails pace, but progress.

[Update]

That would have been too easy tho. On Amazon Linux AMI I seemed to have all sorts of strange permissions problems. properties files couldnt be read, a directory couldnt be created. So I have given up, and have moved back to the ubuntu 10.04 install I have for now. Knowing that the JAVA_OPTS were a problem, and seeing that /etc/tomcat6/tomcat6.conf doesnt exist and is not used by /etc/init.d/tomcat6 on ubuntu, all you need to to get JAVA_OPTS is set them in your env and use sudo with -E.

So thats Tomcat6 running in upto 384m, and Joseki on 256m.  Wonder how they’ll fair…

For log4j, I needed to set a fully qualified path, or I get log create errors.

And then, finally, had to update the Security Group to let traffic thru on the right ports. Now just need to update the code to the latest level, start a fresh repo on Sesame, restart a few more times (sigh) and test the jsp/js code for machine specifics I may have left lying around. Almost there !

Getting UML from java code in Eclipse 3.5

November 8, 2010 Comments off

This should be easier 😦

I had used omondo a couple of years back but trying to install again proved to be too painful. Had to download a complete new eclipse+plugin combo for evaluation purposes and then it wouldnt start – eclipse gave a helpful “error=-1”.

Tried MaintainJ – had looked at a while ago – but couldnt get the plugins to work because my AspectJ code used v7 and it wanted v6. Gave up.

MyEclipseIDE – looks like it has it all including an update url, but it depends on Mozilla XUL 1.9.07 which doesnt seem to be available. Installing 1.9.17 didnt help. Eventually downloaded the the v9 beta (full install of several hundred mB) and imported the projects I needed, right clicked to the MyEclipse menu in the UML2 perspective, and followed the prompts. Right click on diagram to export to JPG. Have not investigated round tripping or sequence diagrams yet – to follow, when I need. But then I restarted it, and it never stops starting. Still I got one diagram from it !

Drupal 7 on Ubuntu 10.10 or 10.04 for SPARQL federation

October 28, 2010 3 comments

Quick steps to install Drupal 7

This, so you can use the SPARQL federating plugin

First install ubuntu ! 10.10 isnt much over 10.04, perhaps less even, so go with what you have v10-wise.

 

Now you need apache,mysql,php and drupal, database setup (ie LAMP + drupal). Then and a variety sparql modules and support.

In short

  1. Install LAMP
  2. Setup MySQL (or create a database for it)
  3. Download and Setup Drupal (or update/upgrade)
  4. Configure Drupal
  5. Add modules

In some more detail

  1. Install Lamp
  2. Install PHP gd – you’ll get errors about it during Drupal install otherwise
  3. Setup mysql with database called d7 and admin user drupal
    1. create database d7
    2. grant all privileges on d7 to drupal@localhost identified by ‘password’ -if you get into trouble install phpmyadmin
  4. download drupal7, expand and move the expanded dir into the docroot/cms (you may need sudo for all of this)
    1. gzip -dc drupal.*.*.tar.gz | tar -xvf -
    2. mkdir /var/www/cms
    3. cd drupal.*.*
    4. mv * /var/www/cms
    5. read the README about creating default site and perms
  5. take a breath
  6. restart apache for the hell of it - sudo /etc/init.d/apache2 restart
  7. go to http://127.0.0.1/cms and create the first site by following the steps
  8. reset perms as per readme on settings.php

now find the SPARQL plugin [1,2] that Lin at DERI created and get to work on content-types[3]……

I’ve left messages but haven’t had a response yet – Sparql Views as it is on GitHub Jan 2011 won’t install on the official Drupal7 release (drush doesnt like it either, it seems). But theres also VARQL[4], built on top of what Lin did, that might be good for you.

There are dependencies you should be prepared to install – Views (not unsurprisingly), Dialog [5], Chaos Tools Suite [6] (maybe  RDFx [7] – Sparql Views has it, needs it, but Drupal7 has some conflict either at install or runtime – needs more investigation).

And you’ll need to download and manually install, as ftp to localhost, using the UI installer won’t work (at least not for me on Ubuntu 10.04/10).  Relying on ftp is just a bad way of installing things IMO.

Update rather than Upgrade

FYI : I easily updated (not upgraded [9]) my Drupal7 Beta to official release byfollowing Yodi Aditya’s instructions [8]

 

 

[1] http://cph2010.drupal.org/sessions/using-rdf-semantic-web-modules

[2] http://drupal.org/project/sparql_views

[3] http://semantic-drupal.com/

[4] http://drupal.org/project/varql

[5] http://drupal.org/project/dialog

[6] http://drupal.org/project/ctools

[7] http://groups.drupal.org/node/59388

[8] http://yoodey.com/how-migration-upgrade-drupal-7-rc-drupal-7-final

[9] http://drupal.org/upgrade

 

Setting up Joseki & TDB with a bunch of RDF files

November 9, 2009 Comments off

Here the idea is that you load a directory of RDF statements into a TDB store and front it with Joseki for Sparql querying.

  • Download and expand Joseki and TDB.
  • Copy the tdb jar file to the Joseki llib folder.
  • Add a servlet mapping in web.xml, copying the books example, for your new dataset. eg newDataServiceRef
  • . You’ll use this in your service description for Joseki, below

  • Copy the query.html to one of your own eg mydata.html, and change the form action to point to the url you just set in web.xml
  • I added the following to the joseki-config.ttl

      Prefix

      @prefix sdb:      <http://jena.hpl.hp.com/2007/sdb#> .
      @prefix tdb:     <http://jena.hpl.hp.com/2008/tdb#> .

      TDB Class load

      ## Datasets
       ## Initialize TDB.
       [] ja:loadClass "com.hp.hpl.jena.tdb.TDB" .
       tdb:DatasetTDB  rdfs:subClassOf  ja:RDFDataset .
       tdb:GraphTDB    rdfs:subClassOf  ja:Model .

      Service

      # Service 3 - SPARQL processor only handling a given dataset
       <#service3>
       rdf:type            joseki:Service ;
       rdfs:label          "My new sparql service" ;
       joseki:serviceRef   "newDataServiceRef" ;   # web.xml must route this name to Joseki
       # dataset part
       joseki:dataset      <#newdata> ;
       # Service part.
       # This processor will not allow either the protocol,
       # nor the query, to specify the dataset.
       joseki:processor    joseki:ProcessorSPARQL_FixedDS ;

      .

      Dataset and Graph

      <#newdata>   rdf:type tdb:DatasetTDB ;
       rdfs:label "A new TDB dataset" ;
       tdb:location "/full/path/to/Data/myJosekiDataDir" ;
       .
       <#graph> rdf:type tdb:GraphTDB ;
       tdb:location "DB" ;
       .
      
  • Start Joseki – make sure Joseki and TDB home are set, wait for the last log message that says the endpoint has started on port 2020
  • export JOSEKIROOT=~/Joseki-3.4.0
    export TDBROOT=~/TDB-0.8.2
  • Load your data in TDB, into your Joseki dataset using tdbload, like this from TDBROOT :
  • bin/tdbloader --loc=/full/path/to/Data/myJosekiDataDir -v /full/path/to/RDFFiles/*.rdf

  • Navigate to your new html page (mydata.html). Try a query – you should get triples back. If you get a message saying “No dataset description for query” then somethings wrong in your joseki-config.ttl.

Semantic Progress

November 7, 2009 Comments off

Cleaned up some of the RDF that I had been trying to ignore, so the code and config is now in a state to do a full run and populate a respository. Getting about 80% hit to GeoNames and about 80% of that again to DBPedia. Will do a manual hitlist later, and work out how to update the repository with decorations like that. Now just have to work out Jena and TDB, and hope my URIs are navigable out of the box – I doubt it.