As some of you may know I work in the QA department at the Venice Project. Up until recently I was the QA department by myself. This is no longer true. We do all our testing manually which is bad - because humans do fail - whereas software is predictable and will always fails the same way.
Up until now we are using Confluence to track our work and our test cases and testings. But this has limits. I would like to add steps to our buildbots build processes to do unit testing and keep track of those - and I would love to keep track of the memory and disk footprint used by our application. There are a few plug-in for confluence that I could use, but they are more oriented in a manual reporting - people click to say if test was successful or not - this is not really compatible with the use of bots to do unit testing. I've done my bits of googling and found many pages describing software that might achieve what I want. I also asked my question on usenet - and up to now had one answer giving me the name of the type of software I'm looking for and a link to a german study/comparison of those pieces of software. Still I would love to get live feedback about the usage of such software so I could make a choice. My needs are :
- Document test cases.
- Write test plans.
- Track test plans - manual testing - .
- Get input data from external source to track what my bots are doing.
- Be Able to track performance for clients (memory, disk footprint).
- Be Able to track performance for Server applications (Stress testing, memory usage, cpu usage)
Integration with svn, Jira are pluses.
If u are looking for a test management tool jira-like, user friendly and designed to work in deep integration with jira, please check http://www.prismasw.it/ita/jite/indexEng.html
You can find features description, data sheet and videotutorials.
Integration with Jira is pretty complete and offers :
Unified Login to Jite and Jira
Automatic update of test status in Jira, due to common id between Jite and Jira
Association of Jite test sessions with created Jira filter, to control from Jira which tests must be executed
Possibility to update from Jite, during the test execution, some custom Jira fields and notify if test has been executed successfully
We have internally implemented a Test Management System using JiRA (managing Requirements, Test Plans, Test Cases, Test Runs and the workflow associated with that), Confluence (only for creating product dashboards), and Test Automation Asset Management System (for Watir) which is integrated with JiRA.
It seems to be working along quite well with our clients really appreciating that.
Brendan
yes that would be nice for testing our backend - but it would not work because it would not be able to build our client. So we will stick with buildbot for the client.
Mads
Thanks for the input. Best whishes for 2007.
Rosie
I have loads to test, so the most I get done by computer bot the more I can test manually - hence My initial question.
Manual testing is good.
If you had to choose between manual or automated approach, what would you go for?
Computers and software fail too - they are created by humans.
I've had the "pleasure" of using Test Director and LoadRunner from mercury and they probably do pretty much what you want. That being said, they're probably expensive enough that they're not worth spending time on.
Have you looked at Atlassian's new product, Bamboo? It is a "continuous integration" (ie "automated build") server which looks like it will develop into something like what you're after. As long as you can write an automated unit test script for your tests, then you should be able to get pretty charts on performance, which tests are always failing etc.
Obviously it supports JIRA and Confluence, it's built by the same people :-)
Good luck with the testing!
Brendan.