xAPI Conformance Testing and Some Thanks

03 December 2014 by Ben Betts

There was some big news in the xAPI world this week, Brian Miller at Rustici recently made an insanely large pull request to ADL’s xAPI test suite on Github (previously it had 1 test, now it has more than 4000).

We’ve been working hard testing Learning Locker to prove (and improve) our compliance. Running the suite has enabled us to finish this work and, after a few days work (with Ryan and Anna putting in some big shifts), we’re pleased to say that we’ve reached 100% conformance with the new tests.

As Brian mentioned in his pull request, the test suite is missing some pieces. However, this is far better than what the xAPI community has right now, and we think it’s a giant leap in the right direction. This will not only be useful for testing the compliance of LRSs but also finding and raising issues with the specification.

Whilst the spec looks comprehensive (and often is) there are areas of conjecture and best practice still to be ironed out. This work is not complete for the xAPI yet and will continue to be a focus for Learning Locker over the coming months.

Upon running the tests we categorised each test into (A) passed, (B) assumptions in the tests, and (C) issues in the tests. You’ll be glad to hear that most of the tests fell into the first of those categories and that we’ve ploughed through any issues to get them fixed, resolving a few outstanding issues on Github in the process.

Tests that fell into the final two categories (B and C) have been documented internally and, once the conformance pull request is merged, we’ll begin to add those as issues on the test suite’s Github repository, where they can be discussed and managed. For now we’ll be maintaining our own modified version of the test suite that Learning Locker passes with a plan to include these tests as part of our continuous integration (via Travis).

Next Steps

If you want to download the tests and run them on Learning Locker, “please be considerate of hosted platforms when running the test suite, it is resource intensive”. We advise you to download and install the issue/conformance branch from our Github repository on your machine, this is currently the most compliant branch with the xAPI specification and the one we’ve been running the tests on.

We now plan to gather feedback about these updates (on the issue/conformance branch), test them and make a few more minor changes with the hope of making a new release some time soon, so stay tuned! As always, you can give us feedback via Github, the Google Group, and Gitter.


We’d like to send a massive thanks to Brian Miller and his team over at Rustici for all the effort they’ve made with these tests. Rustici continue to make great contributions to the Open Source effort that sits behind the xAPI and should be applauded for this latest great piece of work.

I’d also like to put a special mention to Ryan Smith, who, having joined our company just a few months ago, has worked tirelessly to progress the Learning Locker. It is only right that we recognise these efforts in our little meritocracy. As such it gives me great pleasure to welcome Ryan to the Technical Board for Learning Locker. Congratulations Ryan, you deserve every bit of recognition you get.

Ben Betts
Chief Executive Officer

Ben serves as CEO for Learning Pool. Previously, Ben served as Chief Product Officer for Learning Pool where he worked to help define and develop Learning Pool’s next generation of workplace digital learning platforms, with a focus on Learning Experience Platforms and the Learning Analytics space.

Before Learning Pool, Ben helped to build HT2 Labs from humble beginnings into a globally recognized innovator in workplace digital learning.  Learning Pool completed an acquisition of HT2 Labs in June 2019.

Ben’s expertise is based in research, having previously completed his PhD researching the impact of gamification on adult social learning, Ben has authored and contributed chapters for many books, has two peer-reviewed academic papers and has presented at conferences around the world, including TEDx.

Read more about Learning Pool
Visit our Learn and Connect section

Get a free demo

Get in touch to find out how we can help

Start your journey to extraordinary

Your time is valuable, so the more information you can give us, the better we can make your first contact.