xAPI Conformance Testing and Some Thanks

• Home
• Blog
• Learning & Development

Chris Dunne, Digital Marketing Manager

There was some big news in the xAPI world this week, Brian Miller at Rustici recently made an insanely large pull request to ADL’s xAPI test suite on Github (previously it had 1 test, now it has more than 4000).

We’ve been working hard testing Learning Locker to prove (and improve) our compliance. Running the suite has enabled us to finish this work and, after a few days work (with Ryan and Anna putting in some big shifts), we’re pleased to say that we’ve reached 100% conformance with the new tests.

As Brian mentioned in his pull request, the test suite is missing some pieces. However, this is far better than what the xAPI community has right now, and we think it’s a giant leap in the right direction. This will not only be useful for testing the compliance of LRSs but also finding and raising issues with the specification.

Whilst the spec looks comprehensive (and often is) there are areas of conjecture and best practice still to be ironed out. This work is not complete for the xAPI yet and will continue to be a focus for Learning Locker over the coming months.

Upon running the tests we categorised each test into (A) passed, (B) assumptions in the tests, and (C) issues in the tests. You’ll be glad to hear that most of the tests fell into the first of those categories and that we’ve ploughed through any issues to get them fixed, resolving a few outstanding issues on Github in the process.

Tests that fell into the final two categories (B and C) have been documented internally and, once the conformance pull request is merged, we’ll begin to add those as issues on the test suite’s Github repository, where they can be discussed and managed. For now we’ll be maintaining our own modified version of the test suite that Learning Locker passes with a plan to include these tests as part of our continuous integration (via Travis).

Next Steps

If you want to download the tests and run them on Learning Locker, “please be considerate of hosted platforms when running the test suite, it is resource intensive”. We advise you to download and install the issue/conformance branch from our Github repository on your machine, this is currently the most compliant branch with the xAPI specification and the one we’ve been running the tests on.

We now plan to gather feedback about these updates (on the issue/conformance branch), test them and make a few more minor changes with the hope of making a new release some time soon, so stay tuned! As always, you can give us feedback via Github, the Google Group, and Gitter.

Thanks

We’d like to send a massive thanks to Brian Miller and his team over at Rustici for all the effort they’ve made with these tests. Rustici continue to make great contributions to the Open Source effort that sits behind the xAPI and should be applauded for this latest great piece of work.

I’d also like to put a special mention to Ryan Smith, who, having joined our company just a few months ago, has worked tirelessly to progress the Learning Locker. It is only right that we recognise these efforts in our little meritocracy. As such it gives me great pleasure to welcome Ryan to the Technical Board for Learning Locker. Congratulations Ryan, you deserve every bit of recognition you get.