QAing a xulrunner based app : chalenges
Ludovic Hirlimann
Joost QA lead
Plan
- What is Joost
- Tools and processes
- Unit Testing
- End to end testing
- Following outside sources
- Demo
- Question and answer
Visit Joost at http://joost.com and http://opensource.joost.com
Joost™
- Client server
- Tv over the internet
The client is based on Mozilla's XULrunner and uses the ZAP branch. The client combines
this xulrunner with other librarries like the P2P layer - some media libaries etc ...
The backend server side uses Apache bases technologies and are mostly writen in Java.
When QAing such things you need to do end to end testing :
The client
The servers
The communications between both
The data
The metadata
Process and tools
- SVN
- Buildbot
- Confluence
- Jira
[any material that should appear in print but not on the slide]
Jira and confluences are products developed bu Atlassian - and have both
some sort of integration with the other.
Svn is very similar to CVS - but it seems that merging from one branch to
another one is very difficult.
http://electricjellyfish.net/garrett/talks/oscon2004/svn-best-practices/ is a
best practice paper on using SVN.
Buildbot is building our sources on every or so commits.
Unit testing
- Tracking
- Not easy on mozilla's based products
Unit testing is very important for regression testing. Its the best way to catch
big changes that break things.
Challenges here is to match client and back end.
And tracking these - because if you implement unit testing you need to track and decide if a
failure is critical or not. The things are improving because Mozilla is starting to implement
unit testing. So we get that for free. Unfortunatly I did not have time yet to see how usefull
those html oriented test cases might be useful in our case.
End to End testing
- Tracking and reporting
- UI and Non UI
- Other components
End to End testing mean writing Test cases - that's doable
The problem is tracking - most of these needs to be run .
Other issues is you get bored doing that kind of test - so at one point you need
to rest of these. UI is difficult to test - because it needs to scale properly.
Other component are not really testable - But one needs to monitor usage of mem and
disk and CPU.
Following
- Synching
- Reporting upstream bugs
- Fixing some
So the most difficult part of our dev. process and thus QA is that from time to time we need
to get the fixes from mozilla tree - and the new bugs that come with it.
Also this means getting rid of some hacks in the code - making sure nothing breaks when we synch -
but we can't live without synch - so we have a dedicated engeenir even if he is not QA that does loads
of it.
Question and Answers
- Q&A
- http://opensource.joost.com