Object Commando languages, development and design


Strange Loop Talk – Triplestore Testing

I gave a talk yesterday at Strange Loop giving a high level of what I've been working on for the last 6 months. The abstract of the talk was:

Determining what RDF repository to use for a project can be a
daunting task. With so many repository choices, benchmarks and usage
scenarios, where do you start? This talk discusses how Revelytix
answered that question. The talk will cover the test framework
written by Revelytix in Clojure, including a language for defining
tests, a harness for executing the tests and using CouchDB to store
the results. Example Clojure code will be included along with a
discussion around the available RDF benchmarks. The talk will also
discuss the test harness using EC2 instances for cheap performance
testing and how we interpreted those results using Incanter.

It was a short talk, so I went through the material pretty fast, but the slides can be found here.

Tagged as: No Comments

Installing Parliament on Ubuntu

What is Parliament?

Parliament is an open source triple store that is an improved version of DAMLDB. There is some good information on using the triple in the User's Guide. What's interesting about Parliament is how it stores the triples. Relational databases are a more common implementation of an RDF store, but Parliament goes a different way. Parliament takes linked list style of approach It uses BerkeleyDB for storing the URI values and then stores the triple of references in a linked list. For more information on their approach, there's a great paper on it than can be found here.

Building Parliament for Ubuntu

Parliament has binaries on it's website for Windows and Mac, but none for Ubuntu (or any other Linux distro). Parliament is written in C++ and Java, so make sure you have the g++ package and the JDK installed. You'll also need to make sure you have a Subversion client installed to get the source and Ant installed to build the Java code. Below are the steps I went through to get Parliament to run on Linux:

  1. Build Boost Jam
    • Download and unzip Boost Jam
    • Run build.sh in the boost-jam directory
    • Put jam executable in your PATH
  2. Build the Boost C++ Libraries
    • Download and unzip Boost
    • cd into the boost directory and build boost with the following command (modify accordingly if your not on a 64 bit system)
      bjam -q --build-dir=linux/build --stagedir=linux/stage
      --layout=versioned --with-test architecture=combined address-model=64
      variant=debug,release threading=multi link=shared runtime-link=shared
  3. Build and Install BerkleyDB
    • Download and unzip [http://www.oracle.com/technology/software/products/berkeley-db/htdocs/popup/db/4.7.25/db-targz.html BerkleyDB] 4.7.x
    • cd into /build/unix and type ../dist/configure
    • run make
    • run sudo make install
  4. Create the following environment variables (if you don't already have them
    • JAVA_HOME=/usr/lib/jvm/java-6-sun
    • BOOST_ROOT=/path/to/boost/boost_1_42_0
    • BDB_HOME=/usr/local/BerkeleyDB.4.7/
    • BOOST_BUILD_PATH=$BOOST_ROOT/tools/build/v2
    • BOOST_TEST_LOG_LEVEL=message
  5. Building Parliament
    • Checkout the Parliament source: svn checkout --username anonsvn https://projects.semwebcentral.org/svn/parliament/trunk
    • Copy the parliament_dir/doc/Linux/*.jam files to ~/
    • The Parliament build uses pushd and popd, which is not build into /bin/dash (which is where /bin/sh is symlinked in Ubuntu). To fix this, I changed the /bin/sh symlink to /bin/bash
    • Copy build.properties.template from the Parliament source directory to build.properties
    • Comment the various build architectures (for Mac and Windows) and make sure the below line in uncommented nativeBuildParams=toolset=gcc-4.4 address-model=64 variant=release
  6. Source Changes
    • When I tried to build Parliament the first time, I received an error that the method remove could not be found when compiling Parliament/KbCore/FileHandle.cpp. It was due to the lines of code below:
      #if defined(PARLIAMENT_WINDOWS)
      	if (!DeleteFile(filePath.c_str()))
      	if (remove(filePath.c_str()) == -1)

      I added an include to the top of the file:

      #if !defined(PARLIAMENT_WINDOWS)
      #	include <errno.h>
      #	include <fcntl.h>
      #	include <sys/stat.h>
      #      include <stdio.h> //<-- Added this

      to fix the problem.

    • The and build file includes the Mac environment variable for the C libraries, but not the Linux ones. I changed the build.xml in the Parliament directory to include either the Linux or Mac environment variable depending on the build architecture:
      	<condition property="libraryEnvVariable" value="DYLD_LIBRARY_PATH"> <!-- Line 290 -->
      			<os family="mac"/>
      		<condition property="libraryEnvVariable" value="LD_LIBRARY_PATH">
      				<os family="unix"/>
      				<not><os family="mac"/></not>
       <env key="${libraryEnvVariable}" path="${artifactsDir}/${nativeArtifactsDir}"/> <!-- Line 302 -->
  7. From the source tree root, run ant
  8. Copy Parliament-v2.6.7.0-InsertPlatformHere.zip from the target directory to your install directory (can be anywhere)
  9. In the install directory copy all of the files from gcc-4.4/release/64/ to the ParliamentKB directory
  10. Run the StartParliament.sh script to start Parliament