Re: Test lab

Hi everyone,

Here are a couple of additions to the performance test lab
discussion. I hope you will find these useful.

1.) Test tools. The Bristlecone testing package I presented at the
PG Fall 2007 Conference is now available at http://
bristlecone.continuent.org. There are two main tools: Evaluator and
Benchmark. Evaluator generates a CPU-intensive mixed load.
Benchmark generates very specific loads with systematically varying
parameters. I have been using bristlecone to do a lot of testing of
MySQL and PostgreSQL, since we have middleware that runs on both. I
plan to follow Josh's request and run some of the current benchmarks
to compare 8.2.5 vs. 8.3 performance. So far most of my tests have
compared MySQL and PostgreSQL vs. our middleware but I recently
started to compare the databases directly. One initial result:
MySQL appears to be much faster at streaming very large result sets.

2.) Test hardware. We have a number of hosts in Grenoble, France
that are available to help set up a European lab. We gave away 4
to the postgresql.fr folks but if there's anyone else within driving
(or trucking distance) we still have at least a dozen 1U rack
mountable Compaq units. They are in a garage and winter will soon be
upon the Alps, so we need to try to unload them. Unluckily we
overbought hardware in this location but with luck this can be
someone else's good fortune. It probably won't help in the US of A
due to shipping costs.

Please look at Bristlecone. It's very early on but I have found
these tools to be exceedingly useful. Among other things it should
be possible to add features that allow us to do regression testing on
performance, something that is a pain for "normal" test frameworks.

Cheers, Robert

Robert Hodges, CTO, Continuent, Inc.
Email: robert(dot)hodges(at)continuent(dot)com
Mobile: +1-510-501-3728 Skype: hodgesrm

On Nov 6, 2007, at 9:49 AM, Mark Wong wrote:

> On Tue, 06 Nov 2007 13:15:02 +0000
> Simon Riggs <simon(at)2ndquadrant(dot)com> wrote:
>
>> On Mon, 2007-11-05 at 14:33 -0800, Mark Wong wrote:
>>> On 11/4/07, Simon Riggs <simon(at)2ndquadrant(dot)com> wrote:
>>>>
>>>> Why don't you post a TODO list for TPC-E somewhere, so people
>>>> can bite
>>>> small pieces off of the list. I'm sure there's lots of people
>>>> can help
>>>> if we do it that way.
>>>
>>> This should be a good start:
>>>
>>> http://osdldbt.sourceforge.net/dbt5/todo.html
>>>
>>
>> Ah, thanks.
>>
>> Not sure what some of the TODOs mean, but I'll see if I have time to
>> look at some of the code to see if I can help.
>
> No worries, just ask when you get to it. ;) I'm making slow
> progress anyway. I'll get to them all eventually...
>
> Mark
>
> ---------------------------(end of
> broadcast)---------------------------
> TIP 3: Have you checked our extensive FAQ?
>
> http://www.postgresql.org/docs/faq