25 7 / 2014

Anonymous said: Hi Igor! I need your opinion about a CI server which i'm setting up. I'm gonna use it just for automated testing. I use an external app written with Java and Selenium web driver to run different tests to test a webplatform. This app has been programmed tests one by one. Now I need set up Jenkins to run more than one test. Do you think I have to change the external app? or Do you think is better make a multijob build with Jenkins and runn one time per test the external app? Cheers!


There are two was to do this and both have its advantages and disadvantages.

1. You could use a tool like https://github.com/grosser/parallel_tests but for Java, which allows you to run tests in parallel. Then you would create in Jenkins only one job, and run all your tests from it tests will run in parallel due parallel_test tool.

The problem with this option is that your reports will be harder to maintain, since all tests run in one job, and output from console will look weird, since you will have multiple jobs executed at once and the console will show progress of all of them at once, which is going to be hard to read since they will intersect each other due parallelisation.

2. You can use multijob build. This one will require more writing in terms of building the whole suite in Jenkins, however it has its advantages. You could make it run all the tests, and see which ones failed easily, and console output for each will look ok. I did not use multijob yet, but if it can run them at once, you can have parallelisation this way too.

25 7 / 2014

Anonymous said: Hi Igor, My name is Vinoth. I have good experience in functional testing automation with QTP & Selenium. Recently started having interest in performance testing. I am learning JMETER & I have a 'OK' knowledge in it. Your blog helped me to run our test periodically by using jenkins. Thank you so much :) .... 1 question.. We have frameworks like keyword driven, TestNG and all for functional testing ..rt? do we have anything like that for JMeter? Can you plz share your framework / any doc?


I am glad the help articles helped out, you are welcome :). For JMeter I think there is no keyword driven framework. JMeter test can accept parameters by running JMeter from command line, and passing environment variables to it. After that in your test you can decide what you would like to do with environment variables. All of these though have to be written manually from scratch, there are no tools to automate it. Since JMeter is performance testing tool, I think additional framework would slow it down and take more resources for running performance tests.

20 8 / 2013

Performance testing at Postmark

As Postmark is growing, fast and reliable email delivery is becoming a heavier task. At the same time performance testing is more fun. The more emails we are sending, the more details we need to watch out for to preserve the same quality of service.

We have been performance testing Postmark for a long time. Postmark is growing fast and as a result, performance tests are becoming more frequent and more complex.

I would like to share with you a small insight on how we performance test Postmark.

Read More