Percona QA

Merge lp:~dbpercona/percona-qa/readme into lp:percona-qa

  1. readme
  2. Merge into trunk
Proposed by David Bennett
Status: Approved
Approved by: David Bennett
Approved revision: 728
Proposed branch: lp:~dbpercona/percona-qa/readme
Merge into: lp:percona-qa
Diff against target: 1341 lines (+1331/-0)
2 files modified
docs/README.html (+756/-0)
docs/README.md (+575/-0)
To merge this branch: bzr merge lp:~dbpercona/percona-qa/readme
Reviewer Review Type Date Requested Status
Roel Van de Paar (community) Approve
Review via email: mp+234561@code.launchpad.net

Description of the change

Added README markdown and generated HTML version.

To post a comment you must log in.
Revision history for this message
Roel Van de Paar (roel11) wrote :

Hi David,

Thank you for putting this together. I've added you to Percona core, you can now push this. I will edit/update file once pushed.

Thanks

review: Approve
Reply
Revision history for this message
David Bennett (dbpercona) wrote :

Hi Roel,

Yes, I know there are a few updates. Some of the descriptions are from
the usage.

--Dave

On 09/14/2014 03:05 PM, Roel Van de Paar wrote:
> Review: Approve
>
> Hi David,
>
> Thank you for putting this together. I've added you to Percona core, you can now push this. I will edit/update file once pushed.
>
> Thanks

Reply
Revision history for this message
David Bennett (dbpercona) wrote :

Hi Roel,

I opted to just add the README.md to the root directory. My idea with
the docs/ directory is that we could add use case scenarios which I will
collect as we go along and upload as needed. If and when launchpad adds
markdown <https://bugs.launchpad.net/launchpad/+bug/391780> then it
should appear on the website.

I didn't include the README.md as it's a generated file. I used
*discount* which is a markdown to HTML generator.

    $ sudo yum install discount
    $ discount-mkd2html README.md

--Dave

On 09/14/2014 10:31 PM, David Bennett wrote:
> The proposal to merge lp:~dbennett455/percona-qa/readme into lp:percona-qa has been updated.
>
> Status: Needs review => Approved
>
> For more details, see:
> https://code.launchpad.net/~dbennett455/percona-qa/readme/+merge/234561

Reply

Unmerged revisions

728. By David Bennett

Added README describing scripts and utilities

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== added directory 'docs'
2=== added file 'docs/README.html'
3--- docs/README.html 1970-01-01 00:00:00 +0000
4+++ docs/README.html 2014-09-13 05:03:50 +0000
5@@ -0,0 +1,756 @@
6+<!doctype html public "-//W3C//DTD HTML 4.0 Transitional //EN">
7+<html>
8+<head>
9+ <meta name="GENERATOR" content="mkd2html 2.1.6 DEBUG DL=DISCOUNT">
10+ <meta http-equiv="Content-Type"
11+ content="text/html; charset-us-ascii"></head>
12+<body>
13+<h1>Percona QA</h1>
14+
15+<p>Percona QA is a suite of scripts and utilities that assists in building,
16+continuous integration, automated testing and bug reporting for Percona Server
17+and Percona XtraDB Cluster.</p>
18+
19+<h1>Files</h1>
20+
21+<h2>analyze_crashes.sh</h2>
22+
23+<p>This script iterates through MySQL crash core files and runs analyze_crash.sh
24+on them.</p>
25+
26+<h2>analyze_crash.sh</h2>
27+
28+<p>This script takes a MySQL crash core dump as an argument. It creates a an
29+output file with a full backtrace of all threads associated with the core and
30+the time it them backtrace was generated.</p>
31+
32+<h2>analyze_subdir_cores.sh</h2>
33+
34+<p>This script will iterate through all core files starting with the current
35+working directory and creates report containing full backtrace of all threads
36+for each in the file gdb<em><core file name></em>STD/_FULL.txt</p>
37+
38+<h2>build_mysql.sh</h2>
39+
40+<p>Download and build mysql from bzr url.</p>
41+
42+<pre><code>Example: ./build_mysql.sh lp:mysql-server/5.6
43+</code></pre>
44+
45+<p><em>Note:</em> only current cmake builds will work, various compile options can be
46+set by editing the script.</p>
47+
48+<h2>build_percona.sh</h2>
49+
50+<p>download and build percona server from bzr url. Note: script is outdated and
51+it is preferred to use get_percona.sh (see description)</p>
52+
53+<h2>build_xtrabackup.sh</h2>
54+
55+<p>This script will download and build xtrabackup from a bzr url</p>
56+
57+<pre><code>Example: ./build_xtrabackup.sh lp:percona-xtrabackup/2.2
58+</code></pre>
59+
60+<h2>cleanup_ALARMs.sh</h2>
61+
62+<p>Deletes ALARM status RQG trials for which there are no errors of any
63+importance in the server log It does so by checking for which trials ALARM.sh
64+does not return any important failures (Whether this is the case or not is
65+determined by the $FINDS and $IGNOR variables in rqg_results.sh)</p>
66+
67+<p><em>See Also:</em> rqg_failure_counts.sh, rqg_results.sh</p>
68+
69+<h2>cleanup_failures.sh</h2>
70+
71+<p>For use with lp:randgen combinations.pl results</p>
72+
73+<p>Use with caution. Removes all established known issues for a given
74+combinations.pl run based on a search string No valid parameters were passed.
75+Need a status (1) and a search string to cleanup on (2). Retry. Example use:
76+\$cleanup_failures.sh SERVER_CRASHED &lsquo;srv_log_block_size&rsquo; Note; if you want to
77+delete all items for a given status, use &lsquo;a&rsquo; (or shortly: a) as the search
78+string Note; this script relies on the STATUS ./{status passed as first
79+option} scripts to be present Note; this script relies on the
80+delete_single_trial.sh script to be present in the same directory as this
81+script</p>
82+
83+<p><em>See Also:</em> delete_single_trial.sh (calls)</p>
84+
85+<h2>codership6-binary.sh</h2>
86+
87+<p>Build full and package Percona-XtraDB-Cluster-codership.tar.gz binary
88+including the galera engine. Version 5.6</p>
89+
90+<h2>codership-binary.sh</h2>
91+
92+<p>Build full and package Percona-XtraDB-Cluster-codership.tar.gz binary
93+including the galera engine. Version 5.5</p>
94+
95+<h2>COPYING</h2>
96+
97+<p>License (GPL 2)</p>
98+
99+<h2>core_retrieve.sh</h2>
100+
101+<p>For a given trial, this script quickly produces a tarball containing: the core
102+file generated by that trial, the mysqld used by that trial, and the ldd
103+dependency files needed by that This can be used to quickly get a tarball
104+which is ready to be added to a bug report for analysis by developers.</p>
105+
106+<p>Takes the RQG trial number as a parameter.</p>
107+
108+<p><em>See Also:</em> crash_bug_files.sh (called from), ldd_files.sh (calls)</p>
109+
110+<h2>crash_bug_files.sh</h2>
111+
112+<p>This script quickly gathers all files required for logging a crash bug report.</p>
113+
114+<p>Takes the RQG trial number as a parameter.</p>
115+
116+<p><em>See Also:</em> startup.sh (calls), analyze_crash.sh (calls),
117+core_retrieve.sh (calls)</p>
118+
119+<h2>delete_single_trial.sh</h2>
120+
121+<p>For use with lp:randgen combinations.pl results</p>
122+
123+<p>This script deletes a given trial (from a combinations.pl run)
124+completely. Execute from the combinations.pl workdir"</p>
125+
126+<p>Example: to delete trial 1000, execute as:</p>
127+
128+<p>./delete_single_trial.sh 1000"</p>
129+
130+<p><em>See Also:</em> cleanup_ALARMs.sh (called from), cleanup_failures.sh (called from),
131+rqg_results.sh (called from)</p>
132+
133+<h2>doall.sh</h2>
134+
135+<p>This script executes startup.sh in manual RQG mode (without extracting mysql
136+build and randgen), then runs the resulting cmdtrace script and finally
137+prepare_reducer.sh. It expects one parameter: the trial number to analyze</p>
138+
139+<p><em>See Also:</em> startup.sh (calls), prepare_reducer.sh (calls)</p>
140+
141+<h2>docker/</h2>
142+
143+<p>Contained in this directory are multi-node Docker VM configurations PXC
144+testing.</p>
145+
146+<h2>ext/twitter/ip_validator.sh</h2>
147+
148+<p>This script will load and test the MySQL stored function check_ip() into a
149+mysql database. The command line for running the mysql client is defined in
150+the environment variable CLI at the top of the script.</p>
151+
152+<pre><code>check_ip( char(255) ) is an IP address validator.
153+</code></pre>
154+
155+<p><em>The function returns:</em></p>
156+
157+<pre><code>OK (IPV4) - Good IPV4 address
158+OK (IPV6) - Good IPV6 address
159+&gt;!FAIL!(x)&lt; - Fail where 'x' is a-e designating the type of failure
160+ (see function for more info)
161+NOT_AN_IP - Input does not resemble an IPV4 or IPV6 address.
162+</code></pre>
163+
164+<p>The stored function uses the underlying IS_IPV4(), INET_ATON(), INET_NTOA(),
165+INET6_ATON() and INET6_NTOA() functions in order to validate the given ip
166+address.</p>
167+
168+<h2>extract_query.gdb</h2>
169+
170+<p>This is a gdb script which is called from prepare_reducer.sh and helps extract
171+queries that were running from a mysqld core dump.</p>
172+
173+<p><em>See Also:</em> prepare_reducer.sh (called from)</p>
174+
175+<h2>get_percona.sh</h2>
176+
177+<p>This script downloads recent binaries from the Jenkins CI server (optimized,
178+with debug symbols and valgrind) and extracts them onto the local system for
179+testing. This avoids the time required for local builds and issues with
180+toolchain variations. Examine the script to determine or change which
181+OS/target binaries are being downloaded (i.e. centos6-64).</p>
182+
183+<h2>hung_mysqld.gdb</h2>
184+
185+<p>This small gdb command script allows you to analyze the state of a currently
186+running but hung process. It will dump all thread backtrace call stacks for
187+the given process id.</p>
188+
189+<p>Example:</p>
190+
191+<pre><code>gdb mysqld {pid} &lt; hung_mysqld.gdb
192+</code></pre>
193+
194+<h2>keep_single_trial.sh</h2>
195+
196+<p>This script keeps a given RQG trial (from a combinations.pl run) completely
197+(i.e. all files included) in the KEEP directory within a given
198+combinations.pl workdir. Execute this script directly from within the
199+combinations.pl workdir. Example: to keep trial 1000, execute as:</p>
200+
201+<pre><code>./keep_single_trial.sh 1000
202+</code></pre>
203+
204+<h2>ldd_files.sh</h2>
205+
206+<p>This script quickly gathers all shared library dependencies for mysqld and
207+xtrabackup in the current working directory the script is run from. The
208+shared libraries are copied into a <em>lib64</em> subdirectory of the current working
209+directory.</p>
210+
211+<p><em>See Also:</em> core_retrieve.sh (called from)</p>
212+
213+<h2>lp-bug-add-file.py</h2>
214+
215+<p>This script will add a attached file to an existing launchpad bug report.</p>
216+
217+<p>This script needs three parameters; the Launchpad bug number, the file to
218+upload and the file&rsquo;s description.</p>
219+
220+<pre><code>Usage: ./lp-bug-logger.py -b &lt;bugno&gt; -f &lt;file&gt; -d '&lt;description&gt;'
221+</code></pre>
222+
223+<p>This script is usually not executed directly, but is instead used by
224+lp-bug-logger.sh However, if you use it directly, a &lsquo;None&rsquo; output means there
225+were no errors/that the upload was successful.</p>
226+
227+<p>You will need to install <em>python-launchpadlib</em> for the proper operation of
228+this script.</p>
229+
230+<p><em>Ubuntu:</em></p>
231+
232+<pre><code>$sudo apt-get install python-launchpadlib
233+</code></pre>
234+
235+<p><em>Others:</em></p>
236+
237+<pre><code>$sudo pip install launchpadlib
238+</code></pre>
239+
240+<p>Some important information on interaction with Launchpad via the Python
241+Launchpad API used by this script: The first time you run this script, you
242+will see something like this (after quite some time):</p>
243+
244+<pre><code>The authorization page:
245+
246+(https://launchpad.net/+authorize-token?oauth_token=xxxxxxxxxxxxxx&amp;allow_permission=DESKTOP_INTEGRATION)
247+
248+should be opening in your browser. Use your browser to authorize this
249+program to access LP on your behalf Waiting to hear from Launchpad about
250+your decision...
251+</code></pre>
252+
253+<p>When you see this (and assuming you&rsquo;re working in a text-only ssh connection;
254+otherwise a browser may have already automatically opened for you), copy and
255+paste the URL shown into any browser and authorize the app (for example, until
256+revoked) after logging into LP. This only has to be done once (if you chose
257+an indefinite until revoked authorization), and it can be done from any
258+machine (provided you login to LP). If you run into any issues, also review
259+<a href="https://bugs.launchpad.net/launchpadlib/+bug/814595">Unrecoverable error when authorization
260+fails</a> if applicable.
261+And, authorized applications can be viewed (and revoked) at:
262+https://launchpad.net/~&lt;your_lp_user_id>/+oauth-tokens</p>
263+
264+<p><em>See Also:</em> lp-bug-logger.sh (called from)</p>
265+
266+<h2>lp-bug-logger.sh</h2>
267+
268+<p>This script uploads, for a given trial, all files in the directory
269+./BUNDLE_<trialno>/* to a given Launchpad bug report. It expects two
270+parameters; the trial number &amp; the bug number. For Example, to action trial
271+1000, and upload to bug report 1259034 use:</p>
272+
273+<pre><code>$./lp-bug-logger.sh 1000 1259034
274+</code></pre>
275+
276+<p><em>Note:</em> to generate a BUNDLE_<trialno> directory, use crash_bug_files.sh</p>
277+
278+<p><em>Note:</em> This script calls lp-bug-add-file.py, please review the description of
279+that script for other important information.</p>
280+
281+<p><em>See Also:</em> lp-bug-add-file.py (calls)</p>
282+
283+<h2>multirun_cli.sh</h2>
284+
285+<p>Run multiple mysql clients executing the given input SQL multiple times each.</p>
286+
287+<p>This script expects exactly 5 options. Execute as follows:</p>
288+
289+<pre><code>$ multirun_cli.sh threads repetitions input.sql cli_binary socket
290+</code></pre>
291+
292+<table>
293+<thead>
294+<tr>
295+<th style="text-align:left;">Where </th>
296+<th style="text-align:left;"> Description</th>
297+</tr>
298+</thead>
299+<tbody>
300+<tr>
301+<td style="text-align:left;"><em>threads</em> </td>
302+<td style="text-align:left;"> is the number of simultaneously started cli threads</td>
303+</tr>
304+<tr>
305+<td style="text-align:left;"><em>repetitions</em> </td>
306+<td style="text-align:left;"> the number of repetitions <em>per</em> individual thread</td>
307+</tr>
308+<tr>
309+<td style="text-align:left;"><em>input.sql</em> </td>
310+<td style="text-align:left;"> is the input SQL file executed by the clients</td>
311+</tr>
312+<tr>
313+<td style="text-align:left;"><em>cli_binary</em> </td>
314+<td style="text-align:left;"> is the location of the mysql binary</td>
315+</tr>
316+<tr>
317+<td style="text-align:left;"><em>socket</em> </td>
318+<td style="text-align:left;"> is the location of the socket file for connection to mysqld</td>
319+</tr>
320+</tbody>
321+</table>
322+
323+
324+<p><em>Example:</em></p>
325+
326+<pre><code>$ multirun_cli.sh 10 5 bug.sql /ssd/pxc-5.5/bin/mysql /ssd/test/mysql.sock
327+</code></pre>
328+
329+<p><em>Cautionary Notes:</em></p>
330+
331+<ul>
332+<li>Script does not check <em>yet</em> if options passed are all valid/present, make sure you get it right</li>
333+<li>If root user uses a password, a script hack is suggested <em>ftm</em></li>
334+<li>This script may cause very significant server load if used with many threads</li>
335+<li>This script expects write permissions in the current directory (ex. /home/{user}/percona-qa)</li>
336+<li>Output files for each thread are written as: multi.<thread number>.<repetition number> (e.g. multi.1.1 etc.)</li>
337+</ul>
338+
339+
340+<h2>multirun_mysqld.sh</h2>
341+
342+<p>This script will execute a given saved RQG trial against multiple instances of
343+a given mysql installation.</p>
344+
345+<p>This script expects 3 parameters to test server crash with multiple mysqld
346+processes.</p>
347+
348+<p><em>Parameters:</em></p>
349+
350+<ol>
351+<li>mysql base directory</li>
352+<li>RQG trial number</li>
353+<li>crashed SQL file name.</li>
354+</ol>
355+
356+
357+<p><em>Example:</em></p>
358+
359+<pre><code>$ multirun_mysqld.sh /ssd/ramesh/pxc-5.6.20-x64 36 in.sql
360+</code></pre>
361+
362+<p>*See Also:$ myextra.sh (calls)</p>
363+
364+<h2>myextra.sh</h2>
365+
366+<p>This script outputs the extra mysqld options used in a given RQG trial. It
367+expects one parameter: the trial number to analyze. Ensure that startup.sh
368+has already been executed for this trial.</p>
369+
370+<p><em>See Also:</em> multirun_mysqld.sh (called from), prepare_reducer.sh (called from)</p>
371+
372+<h2>parse_general_log.pl</h2>
373+
374+<p>This script can be used to extract SQL from a MySQL general-log.</p>
375+
376+<p><em>Usage:</em></p>
377+
378+<pre><code>[perl] ./parse_general_log.pl -i infile -o outfile
379+</code></pre>
380+
381+<p><em>Where:</em></p>
382+
383+<ul>
384+<li>infile is a standard mysqld general query log</li>
385+<li>outfile is the SQL output extracted from the log.</li>
386+</ul>
387+
388+
389+<p>General mysqld logs can be generated with these options to MTR (or RQG):</p>
390+
391+<pre><code>--mysqld=--log-output=FILE --mysqld=--general_log --mysqld=--general_log_file=out.sql
392+</code></pre>
393+
394+<p><em>WARNING:</em> by default this script eliminates a number of statements which may
395+cause replay issues (KILL, RELEASE) This may lead to non-reproducibility for
396+certain bugs. Check the script&rsquo;s source for specifics</p>
397+
398+<h2>performance.sh</h2>
399+
400+<p>This is a performance testing script that is designed to be run from the
401+Jenkins Continuous Integration system. It will run a series of tests with the
402+MySQL Test Suite and Sysbench both with default and tuned settings.</p>
403+
404+<p>It requires two parameters.</p>
405+
406+<ol>
407+<li>relative workdir</li>
408+<li>relative basedir</li>
409+</ol>
410+
411+
412+<p>Usage example:</p>
413+
414+<pre><code>$./performance.sh 100 Percona-Server-5.5.28-rel29.3-435.Linux.x86_64
415+</code></pre>
416+
417+<p>This would lead to /data/bench/qa/100 being created, in which testing takes
418+/place and data/bench/qa/Percona-Server-5.5.28-rel29.3-435.Linux.x86_64 would
419+/be used to test. The fixed root workdir to which all options are relative is
420+/set to /data/bench/qa in the script.</p>
421+
422+<h2>performance-old.sh</h2>
423+
424+<p>This is an older version of the performance.sh script. It takes the same
425+parameters.</p>
426+
427+<p><em>See:</em> performance.pl</p>
428+
429+<h2>pquery/</h2>
430+
431+<p>A multi-threaded stress test for mysqld. This utility will read SQL queries
432+from a file (one per line) and spawn a given number of threads that will
433+execute the queries from the list in random order. The number of queries each
434+thread will execute is equal to the SQL input line count but it is not
435+guaranteed that all queries will run as they are selected at random, some
436+queries may be run multiple times and others not at all.</p>
437+
438+<p>Also included is a generator script which maybe used to generate queries for
439+pquery. Other tools such as randgen can also be used to generate random query
440+data for pquery input.</p>
441+
442+<p><em>See Also:</em> pquery-sb/</p>
443+
444+<h2>pquery-sb/</h2>
445+
446+<p>pquery-sb is a lua script to be used with the sysbench application. It
447+performs the same function as the C++ based pquery utility.</p>
448+
449+<p><em>See Also:</em> pquery/</p>
450+
451+<h2>prepare_reducer.sh</h2>
452+
453+<p>A wrapper for /util/reducer/reducer.sh in the lp:randgen source tree.</p>
454+
455+<p>This script creates a local run script for reducer. It expects one parameter:
456+the trial number to analyze.</p>
457+
458+<p>To aid with correct bug to testcase generation, this script creates a local
459+run script for reducer and sets #VARMOD#. This handles crashes/asserts for
460+the moment only. Could be expanded later for other cases, and to handle more
461+unforeseen situations.</p>
462+
463+<h2>pxc55to56.sh</h2>
464+
465+<p>2 node cluster testing script, also tests upgrade from 5.5 to 5.6</p>
466+
467+<p>This script tests a two node PXC 5.5 cluster, runs the mysql test suite and
468+sysbench against it, then upgrades the cluster to 5.6 and re-runs the MySQL
469+test suite with and without compatibility options, runs sysbench under 5.6 and
470+then outputs the versions and row count of the test table on both nodes.</p>
471+
472+<h2>pxc56to56.sh</h2>
473+
474+<p>tests cluster combining 5.6 release with 5.6 latest.</p>
475+
476+<p>Tests cluster SST and IST using Mysql test suite and multi-table sysbench from
477+both nodes, performs an mysql_upgrade retests. Reports versions and row
478+counts at the end.</p>
479+
480+<h2>pxc56to56-t2.sh (take 2?)</h2>
481+
482+<p>Alternative version of pxc56to56.sh testing from one release version to
483+another automatically downloads and builds sysbench if not found, wsrep
484+debugging disabled, common parameter ordering changed.</p>
485+
486+<h2>pxc6-bench.sh</h2>
487+
488+<p>This is a benchmark script for Percona XtraDB Cluster 5.6. It benchmarks
489+using MySQL Test Suite, sysbench and has support for dual node cluster
490+testing.</p>
491+
492+<h2>pxc6-rqg.sh</h2>
493+
494+<p>This is an automated randgen testing script for PXC 5.6. Resulting randgen
495+trials are stored for later analysis.</p>
496+
497+<h2>pxc6-sst-test.sh</h2>
498+
499+<p>This script tests Galera snapshot transfer between nodes in a cluster
500+configuration. For PXC 5.6.</p>
501+
502+<h2>pxc-bench.sh</h2>
503+
504+<p>Earlier version of the PXC benchmarking script.</p>
505+
506+<p><em>See:</em> pxc6-bench.sh</p>
507+
508+<h2>pxc-failure-test.sh</h2>
509+
510+<p>This script is for PXC Testing ChaosMonkey Style Please go through <a href="https://github.com/jayjanssen/percona-xtradb-cluster-tutorial">the
511+tutorial</a> to
512+install PXC on your machine and execute this script from node1 Analyze
513+script_out.log file from workdir to get the PXC testcase output</p>
514+
515+<h2>pxc-rqg.sh</h2>
516+
517+<p>Earlier version of the pxc6-rqg.sh randgen script.</p>
518+
519+<p><em>See:</em> pxc6-rqg.sh</p>
520+
521+<h2>pxc-sst-test.sh</h2>
522+
523+<p>Earlier version of the pxc6-sst=test.sh</p>
524+
525+<p><em>See:</em> pxc6-sst-test.sh</p>
526+
527+<h2>rqg_failure_counts.sh</h2>
528+
529+<p>Jenkins CI script to check randgen failure counts.</p>
530+
531+<h2>rqg_results.sh</h2>
532+
533+<p>This script handles various trial outcomes for a given RQG run (run from the
534+RQG combinations.pl workdir containing all trialx.log files)</p>
535+
536+<p><em>Note:</em> this script relies on the delete_single_trial.sh script to be present
537+in the same directory as this script</p>
538+
539+<p>If you start rqg_results.sh with WIPE as the first option ($./rqg_results.sh
540+WIPE) it will wipe all STATUS_ENVIRONMENT_FAILURE, all STATUS_INTERNAL_ERROR,
541+and all STATUS_PERL_FAILURE failed trials by default</p>
542+
543+<h2>run_times_parse.sh</h2>
544+
545+<p>parse_times.sh allows one to check the start and end times for all trials
546+within a RQG combinations.pl run direct This allows for quick analysis to see
547+if any of the trials stalled/hanged/locked up by reviewing the durations</p>
548+
549+<h2>setup_devtoolset.sh</h2>
550+
551+<p><em>Deprecated</em></p>
552+
553+<p>This script does not work, leaving here for future work or future deletion</p>
554+
555+<h2>setup_server.sh</h2>
556+
557+<p>This script makes major changes to any machine it is executed on (installs
558+packages, enables core files, changes significant security settings etc.).
559+Please review the planned changes by reviewing the script contents. All of
560+the changes in the script prepare this machine to be a proper QA server. This
561+script is meant for use on Centos (6.x mainly). Make sure that you want to
562+continue.</p>
563+
564+<p><em>Note:</em> this script removes pulseaudio drivers. If you are using a
565+desktop/would like to keep pulseaudio, please remark the line in the script
566+first. Finally, this script can be executed multiple times without doubling
567+up configurations/without messing up anything. It was specifically coded for
568+this. This script requires sudo and su commands to be available already. Use
569+visudo as described above (in &lsquo;Sudo:&rsquo;) to add yourself to sudo list (faster
570+run). This script is provided AS-IS with no warranty of any kind. Do NOT run
571+on production machines. This script lowers the servers security. Use at own
572+risk.</p>
573+
574+<h3>Initial Setup Reminders (based on Centos minimal cd install):</h3>
575+
576+<table>
577+<thead>
578+<tr>
579+<th style="text-align:left;">Reminder </th>
580+<th style="text-align:left;"> Notes</th>
581+</tr>
582+</thead>
583+<tbody>
584+<tr>
585+<td style="text-align:left;">Syntax: </td>
586+<td style="text-align:left;"> {variable} needs replacing with your specific configuration, for</td>
587+</tr>
588+<tr>
589+<td style="text-align:left;"></td>
590+<td style="text-align:left;"> example {your_port} becomes 22</td>
591+</tr>
592+<tr>
593+<td style="text-align:left;">SSHD: </td>
594+<td style="text-align:left;"> follow http://www.cyberciti.biz/faq/centos-ssh/ except;</td>
595+</tr>
596+<tr>
597+<td style="text-align:left;"></td>
598+<td style="text-align:left;"> Set port to new number in sshd_config and add this line instead in</td>
599+</tr>
600+<tr>
601+<td style="text-align:left;"></td>
602+<td style="text-align:left;"> iptables (adjust port, and adjust your subnet if needed); | -A INPUT</td>
603+</tr>
604+<tr>
605+<td style="text-align:left;"></td>
606+<td style="text-align:left;"> -s 192.168.0.0/24 -m state &ndash;state NEW -m tcp -p tcp &ndash;dport</td>
607+</tr>
608+<tr>
609+<td style="text-align:left;"></td>
610+<td style="text-align:left;"> {your_port} -j ACCEPT |Note there is likely a similar line there</td>
611+</tr>
612+<tr>
613+<td style="text-align:left;"></td>
614+<td style="text-align:left;"> already that needs removing</td>
615+</tr>
616+<tr>
617+<td style="text-align:left;">User: </td>
618+<td style="text-align:left;"> useradd {username} -m -U; su passwd {username}</td>
619+</tr>
620+<tr>
621+<td style="text-align:left;">Disk: </td>
622+<td style="text-align:left;"> use parted for partition management, fdisk -l for partition list</td>
623+</tr>
624+<tr>
625+<td style="text-align:left;">Net: </td>
626+<td style="text-align:left;"> set ONBOOT=yes and BOOTPROTO=dhcp in</td>
627+</tr>
628+<tr>
629+<td style="text-align:left;"></td>
630+<td style="text-align:left;"> /etc/sysconfig/network-scripts/ifcfg-eth0 if using Centos minimal</td>
631+</tr>
632+<tr>
633+<td style="text-align:left;"></td>
634+<td style="text-align:left;"> install CD</td>
635+</tr>
636+<tr>
637+<td style="text-align:left;"></td>
638+<td style="text-align:left;"> also set NETWORKING=yes /etc/sysconfig/network and set a</td>
639+</tr>
640+<tr>
641+<td style="text-align:left;"></td>
642+<td style="text-align:left;"> HOSTNAME={your_host_name}.{your_domain_name} in</td>
643+</tr>
644+<tr>
645+<td style="text-align:left;"></td>
646+<td style="text-align:left;"> /etc/sysconfig/network</td>
647+</tr>
648+<tr>
649+<td style="text-align:left;">Sudo: </td>
650+<td style="text-align:left;"> # visudo (or sudo visudo), then add this line:</td>
651+</tr>
652+<tr>
653+<td style="text-align:left;"></td>
654+<td style="text-align:left;"> {username} ALL=(ALL) NOPASSWD: ALL</td>
655+</tr>
656+<tr>
657+<td style="text-align:left;">fstab: </td>
658+<td style="text-align:left;"> /dev/sdd /ssd /ext4 noatime,nodiratime,discard,errors=remount-ro 0 2 # use fdisk -l | mkfs.ext4</td>
659+</tr>
660+<tr>
661+<td style="text-align:left;">Bzr: </td>
662+<td style="text-align:left;"> to get this script to continue setting up a server once this initial setup is done, use:</td>
663+</tr>
664+<tr>
665+<td style="text-align:left;"></td>
666+<td style="text-align:left;"> sudo yum install bzr; cd ~; bzr branch lp:percona-qa; cd percona-qa; ./setup_server.sh</td>
667+</tr>
668+<tr>
669+<td style="text-align:left;">Bzr setup:</td>
670+<td style="text-align:left;"> [Optional] to configure bzr for uploading code, follow this;</td>
671+</tr>
672+<tr>
673+<td style="text-align:left;"></td>
674+<td style="text-align:left;"> bzr whoami &lsquo;{yourname} &lt;{your.email@somedomain.com}>&rsquo;</td>
675+</tr>
676+<tr>
677+<td style="text-align:left;"></td>
678+<td style="text-align:left;"> bzr whoami # verify it worked</td>
679+</tr>
680+<tr>
681+<td style="text-align:left;"></td>
682+<td style="text-align:left;"> bzr launchpad-login {your_launchpad_id}</td>
683+</tr>
684+<tr>
685+<td style="text-align:left;"></td>
686+<td style="text-align:left;"> vi ~/.ssh/id_rsa.pub # copy this onto one line and paste in \</td>
687+</tr>
688+<tr>
689+<td style="text-align:left;"></td>
690+<td style="text-align:left;"> at: https://launchpad.net/~{your_launchpad_id}/+editsshkeys</td>
691+</tr>
692+<tr>
693+<td style="text-align:left;"></td>
694+<td style="text-align:left;"> Alternatively (alternative to the last line), copy in the</td>
695+</tr>
696+<tr>
697+<td style="text-align:left;"></td>
698+<td style="text-align:left;"> existing &amp; correct ssh key into ~/.ssh</td>
699+</tr>
700+<tr>
701+<td style="text-align:left;"></td>
702+<td style="text-align:left;"> To fix privileges use; cd ~; chmod 755 .ssh; cd .ssh; \</td>
703+</tr>
704+<tr>
705+<td style="text-align:left;"></td>
706+<td style="text-align:left;"> chmod 600 id_rsa; chmod 644 id_rsa.pub; chmod 644 known_hosts</td>
707+</tr>
708+<tr>
709+<td style="text-align:left;"></td>
710+<td style="text-align:left;"> Some launchpad push examples:</td>
711+</tr>
712+<tr>
713+<td style="text-align:left;"></td>
714+<td style="text-align:left;"> bzr push bzr+ssh://bazaar.launchpad.net/~randgen/randgen/rqg2/ &ndash;remember # for RQG + save location</td>
715+</tr>
716+<tr>
717+<td style="text-align:left;"></td>
718+<td style="text-align:left;"> bzr push lp:percona-qa &ndash;remember # for percona-qa + save location</td>
719+</tr>
720+<tr>
721+<td style="text-align:left;"></td>
722+<td style="text-align:left;"> bzr push :parent # for any downloaded R/W branch (handy shortcut), may fail</td>
723+</tr>
724+<tr>
725+<td style="text-align:left;">Bzr repo: </td>
726+<td style="text-align:left;"> Setting up a local repo ensures that broken downloads (removed with rm -Rf {download_dir_name}) can be resumed (a must for PS)</td>
727+</tr>
728+<tr>
729+<td style="text-align:left;"></td>
730+<td style="text-align:left;"> cd /;sudo mkdir bzr;sudo chown -R {username}:{username} bzr;sudo chmod -R 777 bzr;cd bzr;bzr init-repo .</td>
731+</tr>
732+</tbody>
733+</table>
734+
735+
736+<h2>startup.sh</h2>
737+
738+<p>Generate cmd startup scripts used in an RQG trial.</p>
739+
740+<p>This script expects the trial number to be passed - e.g. &lsquo;startup 10&rsquo; for
741+actioning trial 10 (trial10.log) Other uses: &lsquo;startup 10 man&rsquo;: actions trial
742+10 as found in the RQG run without attempting to extract any tars Other uses:
743+&lsquo;startup 0&rsquo;: local build mode (create scripts for a standard server setup
744+without reference to RQG)</p>
745+
746+<p>In other words, use:</p>
747+
748+<pre><code>$ startup.sh &lt;trialno&gt; When analyzing automated Percona Jenkins RQG runs
749+$ startup.sh &lt;trialno&gt; man When analyzing manually executed RQG runs
750+$ startup.sh &lt;trialno&gt; man2 Like man option above, but do not attempt to untar tarball again
751+$ startup.sh 0 When having extracted a binary tar build with ./build/build-binary.sh (no RQG)
752+</code></pre>
753+
754+<h2>test_mysqld_options.sh</h2>
755+
756+<p>This script quickly tests all mysqld options. Note it expects cores to be
757+written to /cores/core.pid. Location can be changed in-script, but not
758+filename To set your server up in this way (in terms of corefile generation),
759+see core file setting part of setup_server.sh, available at lp:percona-qa</p>
760+</body>
761+</html>
762
763=== added file 'docs/README.md'
764--- docs/README.md 1970-01-01 00:00:00 +0000
765+++ docs/README.md 2014-09-13 05:03:50 +0000
766@@ -0,0 +1,575 @@
767+Percona QA
768+==============================================================================
769+
770+Percona QA is a suite of scripts and utilities that assists in building,
771+continuous integration, automated testing and bug reporting for Percona Server
772+and Percona XtraDB Cluster.
773+
774+# Files #
775+
776+## analyze_crashes.sh ##
777+
778+This script iterates through MySQL crash core files and runs analyze_crash.sh
779+on them.
780+
781+## analyze_crash.sh ##
782+
783+This script takes a MySQL crash core dump as an argument. It creates a an
784+output file with a full backtrace of all threads associated with the core and
785+the time it them backtrace was generated.
786+
787+## analyze_subdir_cores.sh ##
788+
789+This script will iterate through all core files starting with the current
790+working directory and creates report containing full backtrace of all threads
791+for each in the file gdb_<core file name>_STD/_FULL.txt
792+
793+## build_mysql.sh ##
794+
795+Download and build mysql from bzr url.
796+
797+ Example: ./build_mysql.sh lp:mysql-server/5.6
798+
799+*Note:* only current cmake builds will work, various compile options can be
800+set by editing the script.
801+
802+## build_percona.sh ##
803+
804+download and build percona server from bzr url. Note: script is outdated and
805+it is preferred to use get_percona.sh (see description)
806+
807+## build_xtrabackup.sh ##
808+
809+This script will download and build xtrabackup from a bzr url
810+
811+ Example: ./build_xtrabackup.sh lp:percona-xtrabackup/2.2
812+
813+## cleanup_ALARMs.sh ##
814+
815+Deletes ALARM status RQG trials for which there are no errors of any
816+importance in the server log It does so by checking for which trials ALARM.sh
817+does not return any important failures (Whether this is the case or not is
818+determined by the $FINDS and $IGNOR variables in rqg_results.sh)
819+
820+*See Also:* rqg_failure_counts.sh, rqg_results.sh
821+
822+## cleanup_failures.sh ##
823+
824+For use with lp:randgen combinations.pl results
825+
826+Use with caution. Removes all established known issues for a given
827+combinations.pl run based on a search string No valid parameters were passed.
828+Need a status (1) and a search string to cleanup on (2). Retry. Example use:
829+\$cleanup_failures.sh SERVER_CRASHED 'srv_log_block_size' Note; if you want to
830+delete all items for a given status, use 'a' (or shortly: a) as the search
831+string Note; this script relies on the STATUS ./{status passed as first
832+option} scripts to be present Note; this script relies on the
833+delete_single_trial.sh script to be present in the same directory as this
834+script
835+
836+*See Also:* delete_single_trial.sh (calls)
837+
838+## codership6-binary.sh ##
839+
840+Build full and package Percona-XtraDB-Cluster-codership.tar.gz binary
841+including the galera engine. Version 5.6
842+
843+## codership-binary.sh ##
844+
845+Build full and package Percona-XtraDB-Cluster-codership.tar.gz binary
846+including the galera engine. Version 5.5
847+
848+## COPYING ##
849+
850+License (GPL 2)
851+
852+## core_retrieve.sh ##
853+
854+For a given trial, this script quickly produces a tarball containing: the core
855+file generated by that trial, the mysqld used by that trial, and the ldd
856+dependency files needed by that This can be used to quickly get a tarball
857+which is ready to be added to a bug report for analysis by developers.
858+
859+Takes the RQG trial number as a parameter.
860+
861+*See Also:* crash_bug_files.sh (called from), ldd_files.sh (calls)
862+
863+## crash_bug_files.sh ##
864+
865+This script quickly gathers all files required for logging a crash bug report.
866+
867+Takes the RQG trial number as a parameter.
868+
869+*See Also:* startup.sh (calls), analyze_crash.sh (calls),
870+core_retrieve.sh (calls)
871+
872+## delete_single_trial.sh ##
873+
874+For use with lp:randgen combinations.pl results
875+
876+This script deletes a given trial (from a combinations.pl run)
877+completely. Execute from the combinations.pl workdir"
878+
879+Example: to delete trial 1000, execute as:
880+
881+./delete_single_trial.sh 1000"
882+
883+*See Also:* cleanup_ALARMs.sh (called from), cleanup_failures.sh (called from),
884+rqg_results.sh (called from)
885+
886+## doall.sh ##
887+
888+This script executes startup.sh in manual RQG mode (without extracting mysql
889+build and randgen), then runs the resulting cmdtrace script and finally
890+prepare_reducer.sh. It expects one parameter: the trial number to analyze
891+
892+*See Also:* startup.sh (calls), prepare_reducer.sh (calls)
893+
894+## docker/ ##
895+
896+Contained in this directory are multi-node Docker VM configurations PXC
897+testing.
898+
899+## ext/twitter/ip_validator.sh ##
900+
901+This script will load and test the MySQL stored function check_ip() into a
902+mysql database. The command line for running the mysql client is defined in
903+the environment variable CLI at the top of the script.
904+
905+ check_ip( char(255) ) is an IP address validator.
906+
907+*The function returns:*
908+
909+ OK (IPV4) - Good IPV4 address
910+ OK (IPV6) - Good IPV6 address
911+ >!FAIL!(x)< - Fail where 'x' is a-e designating the type of failure
912+ (see function for more info)
913+ NOT_AN_IP - Input does not resemble an IPV4 or IPV6 address.
914+
915+The stored function uses the underlying IS_IPV4(), INET_ATON(), INET_NTOA(),
916+INET6_ATON() and INET6_NTOA() functions in order to validate the given ip
917+address.
918+
919+## extract_query.gdb ##
920+
921+This is a gdb script which is called from prepare_reducer.sh and helps extract
922+queries that were running from a mysqld core dump.
923+
924+*See Also:* prepare_reducer.sh (called from)
925+
926+## get_percona.sh ##
927+
928+This script downloads recent binaries from the Jenkins CI server (optimized,
929+with debug symbols and valgrind) and extracts them onto the local system for
930+testing. This avoids the time required for local builds and issues with
931+toolchain variations. Examine the script to determine or change which
932+OS/target binaries are being downloaded (i.e. centos6-64).
933+
934+## hung_mysqld.gdb ##
935+
936+This small gdb command script allows you to analyze the state of a currently
937+running but hung process. It will dump all thread backtrace call stacks for
938+the given process id.
939+
940+Example:
941+
942+ gdb mysqld {pid} < hung_mysqld.gdb
943+
944+## keep_single_trial.sh ##
945+
946+This script keeps a given RQG trial (from a combinations.pl run) completely
947+(i.e. all files included) in the KEEP directory within a given
948+combinations.pl workdir. Execute this script directly from within the
949+combinations.pl workdir. Example: to keep trial 1000, execute as:
950+
951+ ./keep_single_trial.sh 1000
952+
953+## ldd_files.sh ##
954+
955+This script quickly gathers all shared library dependencies for mysqld and
956+xtrabackup in the current working directory the script is run from. The
957+shared libraries are copied into a *lib64* subdirectory of the current working
958+directory.
959+
960+*See Also:* core_retrieve.sh (called from)
961+
962+## lp-bug-add-file.py ##
963+
964+This script will add a attached file to an existing launchpad bug report.
965+
966+This script needs three parameters; the Launchpad bug number, the file to
967+upload and the file's description.
968+
969+ Usage: ./lp-bug-logger.py -b <bugno> -f <file> -d '<description>'
970+
971+This script is usually not executed directly, but is instead used by
972+lp-bug-logger.sh However, if you use it directly, a 'None' output means there
973+were no errors/that the upload was successful.
974+
975+You will need to install *python-launchpadlib* for the proper operation of
976+this script.
977+
978+*Ubuntu:*
979+
980+ $sudo apt-get install python-launchpadlib
981+
982+*Others:*
983+
984+ $sudo pip install launchpadlib
985+
986+Some important information on interaction with Launchpad via the Python
987+Launchpad API used by this script: The first time you run this script, you
988+will see something like this (after quite some time):
989+
990+ The authorization page:
991+
992+ (https://launchpad.net/+authorize-token?oauth_token=xxxxxxxxxxxxxx&allow_permission=DESKTOP_INTEGRATION)
993+
994+ should be opening in your browser. Use your browser to authorize this
995+ program to access LP on your behalf Waiting to hear from Launchpad about
996+ your decision...
997+
998+When you see this (and assuming you're working in a text-only ssh connection;
999+otherwise a browser may have already automatically opened for you), copy and
1000+paste the URL shown into any browser and authorize the app (for example, until
1001+revoked) after logging into LP. This only has to be done once (if you chose
1002+an indefinite until revoked authorization), and it can be done from any
1003+machine (provided you login to LP). If you run into any issues, also review
1004+[Unrecoverable error when authorization
1005+fails](https://bugs.launchpad.net/launchpadlib/+bug/814595) if applicable.
1006+And, authorized applications can be viewed (and revoked) at:
1007+https://launchpad.net/~<your_lp_user_id>/+oauth-tokens
1008+
1009+*See Also:* lp-bug-logger.sh (called from)
1010+
1011+## lp-bug-logger.sh ##
1012+
1013+This script uploads, for a given trial, all files in the directory
1014+./BUNDLE_<trialno>/* to a given Launchpad bug report. It expects two
1015+parameters; the trial number & the bug number. For Example, to action trial
1016+1000, and upload to bug report 1259034 use:
1017+
1018+ $./lp-bug-logger.sh 1000 1259034
1019+
1020+*Note:* to generate a BUNDLE_<trialno> directory, use crash_bug_files.sh
1021+
1022+*Note:* This script calls lp-bug-add-file.py, please review the description of
1023+that script for other important information.
1024+
1025+*See Also:* lp-bug-add-file.py (calls)
1026+
1027+## multirun_cli.sh ##
1028+
1029+Run multiple mysql clients executing the given input SQL multiple times each.
1030+
1031+This script expects exactly 5 options. Execute as follows:
1032+
1033+ $ multirun_cli.sh threads repetitions input.sql cli_binary socket
1034+
1035+Where | Description
1036+:--------------|:------------------------------------------------------------
1037+*threads* | is the number of simultaneously started cli threads
1038+*repetitions* | the number of repetitions *per* individual thread
1039+*input.sql* | is the input SQL file executed by the clients
1040+*cli_binary* | is the location of the mysql binary
1041+*socket* | is the location of the socket file for connection to mysqld
1042+
1043+*Example:*
1044+
1045+ $ multirun_cli.sh 10 5 bug.sql /ssd/pxc-5.5/bin/mysql /ssd/test/mysql.sock
1046+
1047+*Cautionary Notes:*
1048+
1049+- Script does not check *yet* if options passed are all valid/present, make sure you get it right
1050+- If root user uses a password, a script hack is suggested *ftm*
1051+- This script may cause very significant server load if used with many threads
1052+- This script expects write permissions in the current directory (ex. /home/{user}/percona-qa)
1053+- Output files for each thread are written as: multi.<thread number>.<repetition number> (e.g. multi.1.1 etc.)
1054+
1055+## multirun_mysqld.sh ##
1056+
1057+This script will execute a given saved RQG trial against multiple instances of
1058+a given mysql installation.
1059+
1060+This script expects 3 parameters to test server crash with multiple mysqld
1061+processes.
1062+
1063+*Parameters:*
1064+
1065+1. mysql base directory
1066+2. RQG trial number
1067+3. crashed SQL file name.
1068+
1069+*Example:*
1070+
1071+ $ multirun_mysqld.sh /ssd/ramesh/pxc-5.6.20-x64 36 in.sql
1072+
1073+*See Also:$ myextra.sh (calls)
1074+
1075+## myextra.sh ##
1076+
1077+This script outputs the extra mysqld options used in a given RQG trial. It
1078+expects one parameter: the trial number to analyze. Ensure that startup.sh
1079+has already been executed for this trial.
1080+
1081+*See Also:* multirun_mysqld.sh (called from), prepare_reducer.sh (called from)
1082+
1083+## parse_general_log.pl ##
1084+
1085+This script can be used to extract SQL from a MySQL general-log.
1086+
1087+*Usage:*
1088+
1089+ [perl] ./parse_general_log.pl -i infile -o outfile
1090+
1091+*Where:*
1092+
1093+- infile is a standard mysqld general query log
1094+- outfile is the SQL output extracted from the log.
1095+
1096+General mysqld logs can be generated with these options to MTR (or RQG):
1097+
1098+ --mysqld=--log-output=FILE --mysqld=--general_log --mysqld=--general_log_file=out.sql
1099+
1100+*WARNING:* by default this script eliminates a number of statements which may
1101+cause replay issues (KILL, RELEASE) This may lead to non-reproducibility for
1102+certain bugs. Check the script's source for specifics
1103+
1104+## performance.sh ##
1105+
1106+This is a performance testing script that is designed to be run from the
1107+Jenkins Continuous Integration system. It will run a series of tests with the
1108+MySQL Test Suite and Sysbench both with default and tuned settings.
1109+
1110+It requires two parameters.
1111+
1112+1. relative workdir
1113+2. relative basedir
1114+
1115+Usage example:
1116+
1117+ $./performance.sh 100 Percona-Server-5.5.28-rel29.3-435.Linux.x86_64
1118+
1119+This would lead to /data/bench/qa/100 being created, in which testing takes
1120+/place and data/bench/qa/Percona-Server-5.5.28-rel29.3-435.Linux.x86_64 would
1121+/be used to test. The fixed root workdir to which all options are relative is
1122+/set to /data/bench/qa in the script.
1123+
1124+## performance-old.sh ##
1125+
1126+This is an older version of the performance.sh script. It takes the same
1127+parameters.
1128+
1129+*See:* performance.pl
1130+
1131+## pquery/ ##
1132+
1133+A multi-threaded stress test for mysqld. This utility will read SQL queries
1134+from a file (one per line) and spawn a given number of threads that will
1135+execute the queries from the list in random order. The number of queries each
1136+thread will execute is equal to the SQL input line count but it is not
1137+guaranteed that all queries will run as they are selected at random, some
1138+queries may be run multiple times and others not at all.
1139+
1140+Also included is a generator script which maybe used to generate queries for
1141+pquery. Other tools such as randgen can also be used to generate random query
1142+data for pquery input.
1143+
1144+*See Also:* pquery-sb/
1145+
1146+## pquery-sb/ ##
1147+
1148+pquery-sb is a lua script to be used with the sysbench application. It
1149+performs the same function as the C++ based pquery utility.
1150+
1151+*See Also:* pquery/
1152+
1153+## prepare_reducer.sh ##
1154+
1155+A wrapper for /util/reducer/reducer.sh in the lp:randgen source tree.
1156+
1157+This script creates a local run script for reducer. It expects one parameter:
1158+the trial number to analyze.
1159+
1160+To aid with correct bug to testcase generation, this script creates a local
1161+run script for reducer and sets #VARMOD#. This handles crashes/asserts for
1162+the moment only. Could be expanded later for other cases, and to handle more
1163+unforeseen situations.
1164+
1165+## pxc55to56.sh ##
1166+
1167+2 node cluster testing script, also tests upgrade from 5.5 to 5.6
1168+
1169+This script tests a two node PXC 5.5 cluster, runs the mysql test suite and
1170+sysbench against it, then upgrades the cluster to 5.6 and re-runs the MySQL
1171+test suite with and without compatibility options, runs sysbench under 5.6 and
1172+then outputs the versions and row count of the test table on both nodes.
1173+
1174+## pxc56to56.sh ##
1175+
1176+tests cluster combining 5.6 release with 5.6 latest.
1177+
1178+Tests cluster SST and IST using Mysql test suite and multi-table sysbench from
1179+both nodes, performs an mysql_upgrade retests. Reports versions and row
1180+counts at the end.
1181+
1182+## pxc56to56-t2.sh (take 2?) ##
1183+
1184+Alternative version of pxc56to56.sh testing from one release version to
1185+another automatically downloads and builds sysbench if not found, wsrep
1186+debugging disabled, common parameter ordering changed.
1187+
1188+## pxc6-bench.sh ##
1189+
1190+This is a benchmark script for Percona XtraDB Cluster 5.6. It benchmarks
1191+using MySQL Test Suite, sysbench and has support for dual node cluster
1192+testing.
1193+
1194+## pxc6-rqg.sh ##
1195+
1196+This is an automated randgen testing script for PXC 5.6. Resulting randgen
1197+trials are stored for later analysis.
1198+
1199+## pxc6-sst-test.sh ##
1200+
1201+This script tests Galera snapshot transfer between nodes in a cluster
1202+configuration. For PXC 5.6.
1203+
1204+## pxc-bench.sh ##
1205+
1206+Earlier version of the PXC benchmarking script.
1207+
1208+*See:* pxc6-bench.sh
1209+
1210+## pxc-failure-test.sh ##
1211+
1212+This script is for PXC Testing ChaosMonkey Style Please go through [the
1213+tutorial](https://github.com/jayjanssen/percona-xtradb-cluster-tutorial) to
1214+install PXC on your machine and execute this script from node1 Analyze
1215+script_out.log file from workdir to get the PXC testcase output
1216+
1217+## pxc-rqg.sh ##
1218+
1219+Earlier version of the pxc6-rqg.sh randgen script.
1220+
1221+*See:* pxc6-rqg.sh
1222+
1223+## pxc-sst-test.sh ##
1224+
1225+Earlier version of the pxc6-sst=test.sh
1226+
1227+*See:* pxc6-sst-test.sh
1228+
1229+## rqg_failure_counts.sh ##
1230+
1231+Jenkins CI script to check randgen failure counts.
1232+
1233+## rqg_results.sh ##
1234+
1235+This script handles various trial outcomes for a given RQG run (run from the
1236+RQG combinations.pl workdir containing all trialx.log files)
1237+
1238+*Note:* this script relies on the delete_single_trial.sh script to be present
1239+in the same directory as this script
1240+
1241+If you start rqg_results.sh with WIPE as the first option ($./rqg_results.sh
1242+WIPE) it will wipe all STATUS_ENVIRONMENT_FAILURE, all STATUS_INTERNAL_ERROR,
1243+and all STATUS_PERL_FAILURE failed trials by default
1244+
1245+## run_times_parse.sh ##
1246+
1247+parse_times.sh allows one to check the start and end times for all trials
1248+within a RQG combinations.pl run direct This allows for quick analysis to see
1249+if any of the trials stalled/hanged/locked up by reviewing the durations
1250+
1251+## setup_devtoolset.sh ##
1252+
1253+*Deprecated*
1254+
1255+This script does not work, leaving here for future work or future deletion
1256+
1257+## setup_server.sh ##
1258+
1259+This script makes major changes to any machine it is executed on (installs
1260+packages, enables core files, changes significant security settings etc.).
1261+Please review the planned changes by reviewing the script contents. All of
1262+the changes in the script prepare this machine to be a proper QA server. This
1263+script is meant for use on Centos (6.x mainly). Make sure that you want to
1264+continue.
1265+
1266+*Note:* this script removes pulseaudio drivers. If you are using a
1267+desktop/would like to keep pulseaudio, please remark the line in the script
1268+first. Finally, this script can be executed multiple times without doubling
1269+up configurations/without messing up anything. It was specifically coded for
1270+this. This script requires sudo and su commands to be available already. Use
1271+visudo as described above (in 'Sudo:') to add yourself to sudo list (faster
1272+run). This script is provided AS-IS with no warranty of any kind. Do NOT run
1273+on production machines. This script lowers the servers security. Use at own
1274+risk.
1275+
1276+### Initial Setup Reminders (based on Centos minimal cd install): ###
1277+
1278+Reminder | Notes
1279+:---------|:--------------------------------------------------------------------
1280+Syntax: | {variable} needs replacing with your specific configuration, for
1281+ | example {your_port} becomes 22
1282+SSHD: | follow http://www.cyberciti.biz/faq/centos-ssh/ except;
1283+ | Set port to new number in sshd_config and add this line instead in
1284+ | iptables (adjust port, and adjust your subnet if needed); | -A INPUT
1285+ | -s 192.168.0.0/24 -m state --state NEW -m tcp -p tcp --dport
1286+ | {your_port} -j ACCEPT |Note there is likely a similar line there
1287+ | already that needs removing
1288+User: | useradd {username} -m -U; su passwd {username}
1289+Disk: | use parted for partition management, fdisk -l for partition list
1290+Net: | set ONBOOT=yes and BOOTPROTO=dhcp in
1291+ | /etc/sysconfig/network-scripts/ifcfg-eth0 if using Centos minimal
1292+ | install CD
1293+ | also set NETWORKING=yes /etc/sysconfig/network and set a
1294+ | HOSTNAME={your_host_name}.{your_domain_name} in
1295+ | /etc/sysconfig/network
1296+Sudo: | # visudo (or sudo visudo), then add this line:
1297+ | {username} ALL=(ALL) NOPASSWD: ALL
1298+fstab: | /dev/sdd /ssd /ext4 noatime,nodiratime,discard,errors=remount-ro 0 2 # use fdisk -l | mkfs.ext4
1299+Bzr: | to get this script to continue setting up a server once this initial setup is done, use:
1300+ | sudo yum install bzr; cd ~; bzr branch lp:percona-qa; cd percona-qa; ./setup_server.sh
1301+Bzr setup:| [Optional] to configure bzr for uploading code, follow this;
1302+ | bzr whoami '{yourname} <{your.email@somedomain.com}>'
1303+ | bzr whoami # verify it worked
1304+ | bzr launchpad-login {your_launchpad_id}
1305+ | vi ~/.ssh/id_rsa.pub # copy this onto one line and paste in \
1306+ | at: https://launchpad.net/~{your_launchpad_id}/+editsshkeys
1307+ | Alternatively (alternative to the last line), copy in the
1308+ | existing & correct ssh key into ~/.ssh
1309+ | To fix privileges use; cd ~; chmod 755 .ssh; cd .ssh; \
1310+ | chmod 600 id_rsa; chmod 644 id_rsa.pub; chmod 644 known_hosts
1311+ | Some launchpad push examples:
1312+ | bzr push bzr+ssh://bazaar.launchpad.net/~randgen/randgen/rqg2/ --remember # for RQG + save location
1313+ | bzr push lp:percona-qa --remember # for percona-qa + save location
1314+ | bzr push :parent # for any downloaded R/W branch (handy shortcut), may fail
1315+Bzr repo: | Setting up a local repo ensures that broken downloads (removed with rm -Rf {download_dir_name}) can be resumed (a must for PS)
1316+ | cd /;sudo mkdir bzr;sudo chown -R {username}:{username} bzr;sudo chmod -R 777 bzr;cd bzr;bzr init-repo .
1317+
1318+## startup.sh ##
1319+
1320+Generate cmd startup scripts used in an RQG trial.
1321+
1322+This script expects the trial number to be passed - e.g. 'startup 10' for
1323+actioning trial 10 (trial10.log) Other uses: 'startup 10 man': actions trial
1324+10 as found in the RQG run without attempting to extract any tars Other uses:
1325+'startup 0': local build mode (create scripts for a standard server setup
1326+without reference to RQG)
1327+
1328+In other words, use:
1329+
1330+ $ startup.sh <trialno> When analyzing automated Percona Jenkins RQG runs
1331+ $ startup.sh <trialno> man When analyzing manually executed RQG runs
1332+ $ startup.sh <trialno> man2 Like man option above, but do not attempt to untar tarball again
1333+ $ startup.sh 0 When having extracted a binary tar build with ./build/build-binary.sh (no RQG)
1334+
1335+## test_mysqld_options.sh ##
1336+
1337+This script quickly tests all mysqld options. Note it expects cores to be
1338+written to /cores/core.pid. Location can be changed in-script, but not
1339+filename To set your server up in this way (in terms of corefile generation),
1340+see core file setting part of setup_server.sh, available at lp:percona-qa
1341+

Subscribers

People subscribed via source and target branches