Added initial support of stress testing. Now it is possible to start stress... (e4fed851) · Commits · Software / OSDI20 Artifacts / mariadb

mysql-test/README.stress

0 → 100644

+116 −0

Original line number	Diff line number	Diff line

		Overview
		--------

		Stress script is designed to perform testsing of mysql server in
		multi-thread environment.

		Stress script allows:

		- to use for stress testing mysqltest binary as test engine
		- to use for stress testing both regular test suite and any
		additional test suites (e.g. mysql-test-extra-5.0)
		- to specify files with lists of tests both for initialization of
		stress db and for further testing itself
		- to define number of threads that will be concurrently used in testing
		- to define limitations for test run. e.g. number of tests or loops
		for execution or duration of testing, delay between test executions, etc.
		- to get readable log file which can be used for identification of
		errors arose during testing

		All functionality regarding stress testing was implemeted in
		mysql-stress-test.pl script and there are two ways to run stress test:

		- for most cases it is enough to use options below for starting of
		stress test from mysql-test-run wrapper. In this case server will
		be run automatically, all preparation steps will be performed
		and after that stress test will be started.

		- in advanced case one can run mysql-stress-test.pl script directly.
		But it requires to perform some preparation steps and to specify a
		bunch of options as well so this way may look a bit complicate.

		Usage
		-----

		Below is list of stress test specific options for mysql-test-run:

		--stress
		Enable stress mode

		--stress-suite=<suite name>
		Test suite name that will be used in stress testing.
		We assume that all suites are located in mysql-test/suite directory
		There is one special suite name - <main\|default> that corresponds
		to regular test suite located in mysql-test directory.

		--stress-threads=<number of threads>
		Number of threads that will be used in stress testing

		--stress-tests-file=<filename with list of tests>
		Filename with list of tests(without .test suffix) that will be used in
		stress testing. Default filename is stress_tests.txt and default
		location of this file is suite/<suite name>/stress_tests.txt

		--stress-init-file=<filename with list of tests>
		Filename with list of tests(without .test suffix) that will be used in
		stress testing for initialization of stress db. These tests will be
		executed only once before starting of test itself. Default filename
		is stress_init.txt and default location of this file is
		suite/<suite name>/stress_init.txt

		--stress-mode=<method which will be used for choosing tests from the list>
		Possible values are: random(default), seq

		There are two possible modes which affect order of selecting of tests
		from the list:
		- in random mode tests will be selected in random order
		- in seq mode each thread will execute tests in the loop one by one as
		they specified in the list file.

		--stress-test-count= <number>
		Total number of tests that will be executed concurrently by all threads

		--stress-loop-count= <number>
		Total number of loops in seq mode that will be executed concurrently
		by all threads

		--stress-test-duration= <number>
		Duration of stress testing in seconds

		Examples
		--------

		1. Example of simple command line to start stress test:

		mysql-test-run --stress alias

		Runs stress test with default values for number of threads and number of tests,
		with test 'alias' from suite 'main'.

		2. Using in stress testing tests from other suites:

		- mysql-test-run --stress --stress-threads=10 --stress-test-count=1000 \
		--stress-suite=example --stress-tests-file=testslist.txt

		Will run stress test with 10 threads, will execute 1000 tests by all
		threads, test will be used from suite 'example', list of test will be
		taken from file 'testslist.txt'

		- mysql-test-run --stress --stress-threads=10 --stress-test-count=1000 \
		--stress-suite=example sum_distinct

		Will run stress test with 10 threads, will execute 1000 tests by all
		threads, test will be used from suite 'example', list of test contains
		only one test 'sum_distinct'

		3. Debugging of issues found with stress test

		Right now stress test is not fully integrated in mysql-test-run
		and does not support --gdb option so to debug issue found with stress
		test you have to start separately mysql server under debuger and then
		run stress test as:

		- mysql-test-run --extern --stress --stress-threads=10 \
		--stress-test-count=1000 --stress-suite=example \
		sum_distinct

mysql-test/mysql-stress-test.pl

0 → 100755

+1148 −0

File added.

Preview size limit exceeded, changes collapsed.

mysql-test/mysql-test-run.sh

+183 −4

Original line number	Diff line number	Diff line
		@@ -256,6 +256,16 @@ NDB_MGM_EXTRA_OPTS=
		NDB_MGMD_EXTRA_OPTS=
		NDBD_EXTRA_OPTS=

		STRESS_SUITE="main"
		STRESS_MODE="random"
		STRESS_THREADS=5
		STRESS_TEST_COUNT=20
		STRESS_LOOP_COUNT=""
		STRESS_TEST_DURATION=""
		STRESS_INIT_FILE=""
		STRESS_TEST_FILE=""
		STRESS_TEST=""

		while test $# -gt 0; do
		case "$1" in
		--embedded-server)
		@@ -344,6 +354,35 @@ while test $# -gt 0; do
		DO_BENCH=1
		NO_SLAVE=1
		;;
		--stress)
		DO_STRESS=1
		NO_SLAVE=1
		SKIP_SLAVE=1
		;;
		--stress-suite=*)
		STRESS_SUITE=`$ECHO "$1" \| $SED -e "s;--stress-suite=;;"`
		;;
		--stress-threads=*)
		STRESS_THREADS=`$ECHO "$1" \| $SED -e "s;--stress-threads=;;"`
		;;
		--stress-test-file=*)
		STRESS_TEST_FILE=`$ECHO "$1" \| $SED -e "s;--stress-test-file=;;"`
		;;
		--stress-init-file=*)
		STRESS_INIT_FILE=`$ECHO "$1" \| $SED -e "s;--stress-init-file=;;"`
		;;
		--stress-mode=*)
		STRESS_MODE=`$ECHO "$1" \| $SED -e "s;--stress-mode=;;"`
		;;
		--stress-loop-count=*)
		STRESS_LOOP_COUNT=`$ECHO "$1" \| $SED -e "s;--stress-loop-count=;;"`
		;;
		--stress-test-count=*)
		STRESS_TEST_COUNT=`$ECHO "$1" \| $SED -e "s;--stress-test-count=;;"`
		;;
		--stress-test-duration=*)
		STRESS_TEST_DURATION=`$ECHO "$1" \| $SED -e "s;--stress-test-duration=;;"`
		;;
		--big*) # Actually --big-test
		EXTRA_MYSQL_TEST_OPT="$EXTRA_MYSQL_TEST_OPT $1" ;;
		--compress)
		@@ -700,7 +739,7 @@ fi

		# If we should run all tests cases, we will use a local server for that

		if [ -z "$1" ]
		if [ -z "$1" -a $DO_STRESS -eq 0 ]
		then
		USE_RUNNING_SERVER=0
		fi
		@@ -1189,7 +1228,7 @@ start_master()
		then
		CURR_MASTER_MYSQLD_TRACE="$EXTRA_MASTER_MYSQLD_TRACE$1"
		fi
		if [ -z "$DO_BENCH" ]
		if [ -z "$DO_BENCH" -a -z "$DO_STRESS" ]
		then
		master_args="--no-defaults --log-bin=$MYSQL_TEST_DIR/var/log/master-bin$1 \
		--server-id=$id \
		@@ -1626,7 +1665,7 @@ run_testcase ()
		stop_master 1
		report_current_test $tname
		start_master
		if [ -n "$USE_NDBCLUSTER" -a -z "$DO_BENCH" ] ; then
		if [ -n "$USE_NDBCLUSTER" -a -z "$DO_BENCH" -a -z "$DO_STRESS" ] ; then
		start_master 1
		fi
		TZ=$MY_TZ; export TZ
		@@ -1642,7 +1681,7 @@ run_testcase ()
		stop_master 1
		report_current_test $tname
		start_master
		if [ -n "$USE_NDBCLUSTER" -a -z "$DO_BENCH" ] ; then
		if [ -n "$USE_NDBCLUSTER" -a -z "$DO_BENCH" -a -z "$DO_STRESS" ] ; then
		start_master 1
		fi
		else
		@@ -1763,6 +1802,120 @@ run_testcase ()
		fi
		}

		run_stress_test()
		{

		STRESS_BASEDIR="$MYSQL_TEST_DIR/var/stress"

		#Clean-up old stress test basedir
		if [ -d $STRESS_BASEDIR ] ; then
		$RM -rf $STRESS_BASEDIR
		fi
		#Create stress test basedir
		mkdir $STRESS_BASEDIR

		if [ "$STRESS_SUITE" != "main" -a "$STRESS_SUITE" != "default" ] ; then
		STRESS_SUITE_DIR="$MYSQL_TEST_DIR/suite/$STRESS_SUITE"
		else
		STRESS_SUITE_DIR="$MYSQL_TEST_DIR"
		fi

		if [ -d "$STRESS_SUITE_DIR" ] ; then
		STRESS_SUITE_T_DIR="$STRESS_SUITE_DIR/t"
		STRESS_SUITE_R_DIR="$STRESS_SUITE_DIR/r"
		#FIXME: check that dirs above are exist
		else
		echo "Directory $STRESS_SUITE_DIR with test suite doesn't exists. Abort stress testing"
		exit 1
		fi

		if [ -n "$STRESS_TEST" ] ; then
		STRESS_TEST_FILE="$STRESS_BASEDIR/stress_tests.txt"
		echo $STRESS_TEST > $STRESS_TEST_FILE
		elif [ -n "$STRESS_TEST_FILE" ] ; then
		STRESS_TEST_FILE="$STRESS_SUITE_DIR/$STRESS_TEST_FILE"
		if [ ! -f "$STRESS_TEST_FILE" ] ; then
		echo "Specified file $STRESS_TEST_FILE with list of tests does not exist. Please ensure that file exists and has proper permissions"
		exit 1
		fi
		else
		STRESS_TEST_FILE="$STRESS_SUITE_DIR/stress_tests.txt"
		if [ ! -f "$STRESS_TEST_FILE" ] ; then
		echo "Default file $STRESS_TEST_FILE with list of tests does not exist. Please use --stress-test-file option to specify custom one or you can just specify name of test for testing as last argument in command line"
		exit 1
		fi
		fi

		if [ -n "$STRESS_INIT_FILE" ] ; then
		STRESS_INIT_FILE="$STRESS_SUITE_DIR/$STRESS_INIT_FILE"
		if [ ! -f "$STRESS_INIT_FILE" ] ; then
		echo "Specified file $STRESS_INIT_FILE with list of tests doesn't exist. Please ensure that file exists and has proper permissions"
		exit 1
		fi
		else
		STRESS_INIT_FILE="$STRESS_SUITE_DIR/stress_init.txt"
		#Check for default init file
		if [ ! -f "$STRESS_INIT_FILE" ] ; then
		STRESS_INIT_FILE=""
		fi
		fi

		if [ "$STRESS_MODE" != "random" -a "$STRESS_MODE" != "seq" ] ; then
		echo "You specified wrong mode '$STRESS_MODE' for stress test. Correct values are 'random' or 'seq'"
		exit 1
		fi

		STRESS_TEST_ARGS="--server-socket=$MASTER_MYSOCK \
		--server-user=$DBUSER \
		--server-database=$DB \
		--stress-suite-basedir=$MYSQL_TEST_DIR \
		--suite=$STRESS_SUITE \
		--stress-tests-file=$STRESS_TEST_FILE \
		--stress-basedir=$STRESS_BASEDIR \
		--server-logs-dir=$STRESS_BASEDIR \
		--stress-mode=$STRESS_MODE \
		--mysqltest=$BASEDIR/client/mysqltest \
		--threads=$STRESS_THREADS \
		--verbose \
		--cleanup \
		--log-error-details \
		--abort-on-error"

		if [ -n "$STRESS_INIT_FILE" ] ; then
		STRESS_TEST_ARGS="$STRESS_TEST_ARGS --stress-init-file=$STRESS_INIT_FILE"
		fi

		if [ -n "$STRESS_LOOP_COUNT" ] ; then
		STRESS_TEST_ARGS="$STRESS_TEST_ARGS --loop-count=$STRESS_LOOP_COUNT"
		fi

		if [ -n "$STRESS_TEST_COUNT" ] ; then
		STRESS_TEST_ARGS="$STRESS_TEST_ARGS --test-count=$STRESS_TEST_COUNT"
		fi

		if [ -n "$STRESS_TEST_DURATION" ] ; then
		STRESS_TEST_ARGS="$STRESS_TEST_ARGS --test-duration=$STRESS_TEST_DURATION"
		fi

		echo "Stress test related variables:"
		echo "TESTS - $1"
		echo "STRESS - $DO_STRESS"
		echo "STRESS_SUITE - $STRESS_SUITE"
		echo "STRESS_TEST_FILE - $STRESS_TEST_FILE"
		echo "STRESS_INIT_FILE - $STRESS_INIT_FILE"
		echo "STRESS_THREADS - $STRESS_THREADS"
		echo "STRESS_MODE - $STRESS_MODE"
		echo "STRESS_TEST_COUNT - $STRESS_TEST_COUNT"
		echo "STRESS_LOOP_COUNT - $STRESS_LOOP_COUNT"
		echo "STRESS_TEST_DURATION - $STRESS_TEST_DURATION"

		#echo "$STRESS_TEST_ARGS";
		#Run stress test
		$MYSQL_TEST_DIR/mysql-stress-test.pl $STRESS_TEST_ARGS


		}

		######################################################################
		# Main script starts here
		######################################################################
		@@ -1877,6 +2030,32 @@ then
		exit
		fi

		#
		# Stress testing
		#
		if [ "$DO_STRESS" = 1 ]
		then

		if [ -n "$1" ] ; then
		STRESS_TEST="$1";
		fi

		if [ $USE_RUNNING_SERVER -eq 0 ] ; then
		start_master
		fi

		run_stress_test

		if [ $USE_RUNNING_SERVER -eq 0 ] ; then
		mysql_stop
		stop_manager
		fi

		exit

		fi


		$ECHO
		if [ x$USE_TIMER = x1 ] ; then
		$ECHO "TEST RESULT TIME (ms)"