Commit 19bbb7cc authored by unknown's avatar unknown
Browse files

WL #2604: Partition Management 

Optimised version of ADD/DROP/REORGANIZE partitions for
non-NDB storage engines.
New syntax to handle REBUILD/OPTIMIZE/ANALYZE/CHECK/REPAIR partitions
Quite a few bug fixes


include/thr_lock.h:
  New method to downgrade locks from TL_WRITE_ONLY
  Possibility to upgrade lock while aborting locks
mysql-test/r/ndb_autodiscover.result:
  Fix for lowercase and that all NDB tables are now partitioned
mysql-test/r/ndb_bitfield.result:
  Fix for lowercase and that all NDB tables are now partitioned
mysql-test/r/ndb_gis.result:
  Fix for lowercase and that all NDB tables are now partitioned
mysql-test/r/ndb_partition_key.result:
  New test case
mysql-test/r/partition.result:
  New test case
mysql-test/r/partition_error.result:
  New test case
mysql-test/r/partition_mgm_err.result:
  Fix of test case results
mysql-test/t/disabled.def:
  partition_03ndb still has bug
mysql-test/t/ndb_partition_key.test:
  New test cases for new functionality and bugs
mysql-test/t/partition.test:
  New test cases for new functionality and bugs
mysql-test/t/partition_error.test:
  New test cases for new functionality and bugs
mysql-test/t/partition_mgm_err.test:
  New test cases for new functionality and bugs
mysys/thr_lock.c:
  New method to downgrade TL_WRITE_ONLY locks
  Possibility to specify if locks are to be upgraded at abort locks
sql/ha_archive.cc:
  New handlerton methods
sql/ha_berkeley.cc:
  New handlerton methods
sql/ha_blackhole.cc:
  New handlerton methods
sql/ha_federated.cc:
  New handlerton methods
sql/ha_heap.cc:
  New handlerton methods
sql/ha_innodb.cc:
  New handlerton methods
sql/ha_myisam.cc:
  New handlerton methods
sql/ha_myisammrg.cc:
  New handlerton methods
sql/ha_ndbcluster.cc:
  New handlerton methods
  Moved out packfrm and unpackfrm methods
  Adapted many parts to use table_share instead of table->s
  Ensured that .ndb file uses filename and not tablename
  according to new encoding of names (WL 1324)
  All NDB tables are partitioned and set up partition info
  Fixed such that tablenames use tablenames and not filenames in NDB
  NDB uses auto partitioning for ENGINE=NDB tables
  Warning for very large tables
  Set RANGE data
  Set LIST data
  New method to set-up partition info
  Set Default number of partitions flag
  Set linear hash flag
  Set node group array
  Set number of fragments
  Set max rows
  Set tablespace names
  New method to get number of partitions of table to use at open table
sql/ha_ndbcluster.h:
  Removed partition_flags and alter_table_flags from handler class
  A couple of new and changed method headers
sql/ha_ndbcluster_binlog.cc:
  Use new method headers
sql/ha_partition.cc:
  New handlerton methods
  Lots of new function headers
  Use #P# as separator between table name and partition name and
  #SP# as separator between partition name and subpartition name
  Use filename encoding for files both of table name part and of
  partition name parts
  New method to drop partitions based on partition state
  New method to rename partitions based on partition state
  New methods to optimize, analyze, check and repair partitions
  New methods to optimize, analyze, check and repair table
  Helper method to create new partition, open it and external lock
  it, not needed to lock it internally since no one else knows about
  it yet.
  Cleanup method at error for new partitions
  New methods to perform bulk of work at ADD/REORGANIZE partitions
  (change_partitions, copy_partitions)
sql/ha_partition.h:
  New methods and variables
  A few dropped ones and a few changed ones
sql/handler.cc:
  Handlerton interface changes
  New flag to open_table_from_share
sql/handler.h:
  New alter_table_flags
  New partition flags
  New partition states
  More states for default handling
  Lots of new, dropped and changed interfaces
sql/lex.h:
  Added REBUILD and changed name of REORGANISE to REORGANIZE
sql/lock.cc:
  Method to downgrade locks
  Able to specify if locks upgraded on abort locks
sql/log.cc:
  New handlerton methods
sql/mysql_priv.h:
  Lots of new interfaces
sql/share/errmsg.txt:
  Lots of new, dropped and changed error messages
sql/sql_base.cc:
  Adapted to new method headers
  New method to abort and upgrade lock
  New method to close open tables and downgrade lock
  New method to wait for completed table
sql/sql_lex.h:
  New flags
sql/sql_partition.cc:
  Return int instead of bool in get_partition_id
  More defaults handling
  Make use of new mem_alloc_error method
  More work on function headers
  Changes to generate partition syntax to cater for intermediate
  partition states
  Lots of new code with large comments describing new features for
  Partition Management:
  ADD/DROP/REORGANIZE/OPTIMIZE/ANALYZE/CHECK/REPAIR partitions
sql/sql_show.cc:
  Minors
sql/sql_table.cc:
  Moved a couple of methods
  New methods to copy create lists and key lists
  for use with mysql_prepare_table
  New method to write frm file
  New handling of handlers with auto partitioning
  Fix CREATE TABLE LIKE
  Moved code for ADD/DROP/REORGANIZE partitions
  Use handlerton method for alter_table_flags
sql/sql_yacc.yy:
  More memory alloc error checks
  New syntax for REBUILD, ANALYZE, CHECK, OPTIMIZE, REPAIR partitions
sql/table.cc:
  Fix length of extra part to be 4 bytes
  Partition state introduced in frm file
sql/table.h:
  Partition state introduced
sql/unireg.cc:
  Partition state introduced
  Default partition
storage/csv/ha_tina.cc:
  New handlerton methods
storage/example/ha_example.cc:
  New handlerton methods
storage/ndb/include/kernel/ndb_limits.h:
  RANGE DATA
storage/ndb/include/kernel/signaldata/AlterTable.hpp:
  New interfaces in ALTER TABLE towards NDB kernel
storage/ndb/include/kernel/signaldata/DiAddTab.hpp:
  New section
storage/ndb/include/kernel/signaldata/DictTabInfo.hpp:
  Lots of new parts of table description
storage/ndb/include/kernel/signaldata/LqhFrag.hpp:
  tablespace id specified in LQHFRAGREQ
storage/ndb/include/ndbapi/NdbDictionary.hpp:
  Lots of new methods in NDB dictionary
storage/ndb/src/common/debugger/signaldata/DictTabInfo.cpp:
  Lots of new variables in table description
storage/ndb/src/kernel/blocks/dbdict/Dbdict.cpp:
  Lots of new variables in table description
storage/ndb/src/kernel/blocks/dbdict/Dbdict.hpp:
  Lots of new variables in table description
storage/ndb/src/kernel/blocks/dblqh/DblqhMain.cpp:
  New error insertion
storage/ndb/src/kernel/blocks/dbtup/DbtupMeta.cpp:
  a few extra jam's
storage/ndb/src/ndbapi/NdbBlob.cpp:
  Changes to definition of blob tables
storage/ndb/src/ndbapi/NdbDictionary.cpp:
  Lots of new stuff in NDB dictionary
storage/ndb/src/ndbapi/NdbDictionaryImpl.cpp:
  Lots of new stuff in NDB dictionary
storage/ndb/src/ndbapi/NdbDictionaryImpl.hpp:
  Lots of new stuff in NDB dictionary
storage/ndb/test/ndbapi/test_event.cpp:
  removed use of methods no longer in existence
storage/ndb/tools/restore/Restore.cpp:
  Renamed variable
parent d712a1eb

include/thr_lock.h

+3 −1
Original line number Diff line number Diff line
@@ -143,10 +143,12 @@ void thr_unlock(THR_LOCK_DATA *data); 
enum enum_thr_lock_result thr_multi_lock(THR_LOCK_DATA **data,

                                         uint count, THR_LOCK_OWNER *owner);

void thr_multi_unlock(THR_LOCK_DATA **data,uint count);

void thr_abort_locks(THR_LOCK *lock);

void thr_abort_locks(THR_LOCK *lock, bool upgrade_lock);

my_bool thr_abort_locks_for_thread(THR_LOCK *lock, pthread_t thread);

void thr_print_locks(void);		/* For debugging */

my_bool thr_upgrade_write_delay_lock(THR_LOCK_DATA *data);

void    thr_downgrade_write_lock(THR_LOCK_DATA *data,

                                 enum thr_lock_type new_lock_type);

my_bool thr_reschedule_write_lock(THR_LOCK_DATA *data);

#ifdef	__cplusplus

}

mysql-test/include/partition_1.inc

0 → 100644
+747 −0

File added.

Preview size limit exceeded, changes collapsed.

mysql-test/include/partition_10.inc

0 → 100644
+73 −0
Original line number Diff line number Diff line 
# include/partition_10.inc

#

#     Do some basic checks on a table.

#

# FIXME: Do not write the statements and results, if SQL return code = 0

#        and result set like expected. Write a message, that all is like

#        expected instead.

#

# All SELECTs are so written, that we get my_value = 1, when everything

# is like expected.

#



--source include/partition_layout.inc



####### Variations with multiple records

#      Select on empty table

SELECT COUNT(*) = 0 AS my_value FROM t1;

#      (mass) Insert of $max_row records

eval INSERT INTO t1 SELECT * FROM t0_template WHERE f1 BETWEEN 1 AND $max_row;

#      Select

eval SELECT (COUNT(*) = $max_row) AND (MIN(f1) = 1) AND (MAX(f1) = $max_row)

            AS my_value FROM t1;

# DEBUG SELECT COUNT(*),MIN(f1),MAX(f1) FROM t1;

#      (mass) Update $max_row_div4 * 2 + 1 records

eval UPDATE t1 SET f1 = f1 + $max_row

WHERE f1 BETWEEN $max_row_div2 - $max_row_div4 AND $max_row_div2 + $max_row_div4;

#      Select

eval SELECT (COUNT(*) = $max_row) AND (MIN(f1) = 1) AND (MAX(f1) = $max_row_div2 + $max_row_div4 + $max_row )

            AS my_value FROM t1;

# DEBUG SELECT COUNT(*),MIN(f1),MAX(f1) FROM t1;

#      (mass) Delete $max_row_div4 * 2 + 1 records

eval DELETE FROM t1

WHERE f1 BETWEEN $max_row_div2 - $max_row_div4 + $max_row AND $max_row_div2 + $max_row_div4 + $max_row;

#      Select

eval SELECT (COUNT(*) = $max_row - $max_row_div4 - $max_row_div4 - 1) AND (MIN(f1) = 1) AND (MAX(f1) = $max_row)

            AS my_value FROM t1;

# DEBUG SELECT COUNT(*),MIN(f1),MAX(f1) FROM t1;



####### Variations with single records

#      Insert one record at beginning

INSERT INTO t1 SET f1 = 0 , f2 = '#######';

#      Select this record

SELECT COUNT(*) = 1 AS my_value FROM t1 WHERE f1 = 0 AND f2 = '#######';

#      Insert one record at end

eval INSERT INTO t1 SET f1 = $max_row + 1, f2 = '#######';

#      Select this record

eval SELECT COUNT(*) = 1 AS my_value FROM t1 WHERE f1 = $max_row + 1 AND f2 = '#######';

#      Update one record

eval UPDATE t1 SET f1 = $max_row + 2, f2 = 'ZZZZZZZ'

     WHERE  f1 = 0 AND f2 = '#######';

#      Select

eval SELECT COUNT(*) = 1 AS my_value FROM t1 WHERE f1 = $max_row + 2 AND f2 = 'ZZZZZZZ';

if ($fixed_bug15968)

{

# Bug #15968: Partitions: crash when INSERT with f1 = -1 into PARTITION BY HASH(f1)

eval UPDATE t1 SET f1 = 0 - 1, f2 = 'ZZZZZZZ'

     WHERE  f1 = $max_row + 1 AND f2 = '#######';

#      Select

SELECT COUNT(*) AS my_value FROM t1 WHERE f1 = 0 - 1 AND f2 = 'ZZZZZZZ';

}

#      Delete

eval DELETE FROM t1 WHERE f1 = $max_row + 2 AND f2 = 'ZZZZZZZ';

if ($fixed_bug15968)

{

DELETE FROM t1 WHERE f1 = 0 - 1 AND f2 = 'ZZZZZZZ';

}

#      Select

SELECT COUNT(*) = 0 AS my_value FROM t1 WHERE f2 = 'ZZZZZZZ';



#      Truncate

TRUNCATE t1;

#      Select on empty table

SELECT COUNT(*) = 0 AS my_value FROM t1;

mysql-test/include/partition_11.inc

0 → 100644
+34 −0
Original line number Diff line number Diff line 
# include/partition_11.inc

#

#    Try to create a table with the given partition number

#



eval CREATE TABLE t1 ( f1 INTEGER, f2 char(20))

PARTITION BY HASH(f1) PARTITIONS $part_number;

--disable_query_log

eval SET @my_errno= $mysql_errno ;

let $run= `SELECT @my_errno = 0`;

--enable_query_log

#

# If this operation was successfull, check + drop this table

if ($run)

{

   --source include/partition_10.inc

   eval DROP TABLE t1;

}

#### Try to create a table with the given subpartition number

eval CREATE TABLE t1 ( f1 INTEGER, f2 char(20))

PARTITION    BY RANGE(f1) SUBPARTITION BY HASH(f1)

SUBPARTITIONS $part_number

(PARTITION part1 VALUES LESS THAN ($max_row_div2), PARTITION part2 VALUES LESS THAN ($max_int_4));

--disable_query_log

eval SET @my_errno= $mysql_errno ;

let $run= `SELECT @my_errno = 0`;

--enable_query_log

#

# If this operation was successfull, check + drop this table

if ($run)

{

   --source include/partition_10.inc

   eval DROP TABLE t1;

}

mysql-test/include/partition_12.inc

0 → 100644
+65 −0
Original line number Diff line number Diff line 
# include/partition_12.inc

#

# Do some basic things on a table, if the SQL command executed just before

# sourcing this file was successful.

#



--source include/partition_layout.inc



####### Variations with multiple records

   #      (mass) Insert max_row_div2 + 1 records

   eval INSERT INTO t1 SELECT * FROM t0_template WHERE f1 BETWEEN $max_row_div2 AND $max_row;

   #      Select

   eval SELECT (COUNT(*) = $max_row) AND (MIN(f1) = 1) AND (MAX(f1) = $max_row)

            AS my_value FROM t1;

   # DEBUG SELECT COUNT(*),MIN(f1),MAX(f1) FROM t1;

   #      (mass) Update $max_row_div4 * 2 + 1 records

   eval UPDATE t1 SET f1 = f1 + $max_row

   WHERE f1 BETWEEN $max_row_div2 - $max_row_div4 AND $max_row_div2 + $max_row_div4;

   #      Select

   eval SELECT (COUNT(*) = $max_row) AND (MIN(f1) = 1) AND (MAX(f1) = $max_row_div2 + $max_row_div4 + $max_row )

            AS my_value FROM t1;

   # DEBUG SELECT COUNT(*),MIN(f1),MAX(f1) FROM t1;

   #      (mass) Delete $max_row_div4 * 2 + 1 records

   eval DELETE FROM t1

   WHERE f1 BETWEEN $max_row_div2 - $max_row_div4 + $max_row AND $max_row_div2 + $max_row_div4 + $max_row;

   #      Select

   eval SELECT (COUNT(*) = $max_row - $max_row_div4 - $max_row_div4 - 1) AND (MIN(f1) = 1) AND (MAX(f1) = $max_row)

            AS my_value FROM t1;

   # DEBUG SELECT COUNT(*),MIN(f1),MAX(f1) FROM t1;



####### Variations with single records

#      Insert one record at beginning

INSERT INTO t1 SET f1 = 0 , f2 = '#######';

#      Select this record

SELECT COUNT(*) = 1 AS my_value FROM t1 WHERE f1 = 0 AND f2 = '#######';

#      Insert one record at end

eval INSERT INTO t1 SET f1 = $max_row + 1, f2 = '#######';

#      Select this record

eval SELECT COUNT(*) = 1 AS my_value FROM t1 WHERE f1 = $max_row + 1 AND f2 = '#######';

#      Update one record

eval UPDATE t1 SET f1 = $max_row + 2, f2 = 'ZZZZZZZ'

     WHERE  f1 = 0 AND f2 = '#######';

#      Select

eval SELECT COUNT(*) = 1 AS my_value FROM t1 WHERE f1 = $max_row + 2 AND f2 = 'ZZZZZZZ';

if ($fixed_bug15968)

{

# Bug #15968: Partitions: crash when INSERT with f1 = -1 into PARTITION BY HASH(f1)

eval UPDATE t1 SET f1 = 0 - 1, f2 = 'ZZZZZZZ'

     WHERE  f1 = $max_row + 1 AND f2 = '#######';

#      Select

SELECT COUNT(*) AS my_value FROM t1 WHERE f1 = 0 - 1 AND f2 = 'ZZZZZZZ';

}

#      Delete

eval DELETE FROM t1 WHERE f1 = $max_row + 2 AND f2 = 'ZZZZZZZ';

if ($fixed_bug15968)

{

DELETE FROM t1 WHERE f1 = 0 - 1 AND f2 = 'ZZZZZZZ';

}

#      Select

SELECT COUNT(*) = 0 AS my_value FROM t1 WHERE f2 = 'ZZZZZZZ';



#      Truncate

TRUNCATE t1;

#      Select on empty table

SELECT COUNT(*) = 0 AS my_value FROM t1;