com.bigdata.btree.raba.codec

Class AbstractRabaCoderTestCase

java.lang.Object

junit.framework.Assert

junit.framework.TestCase

junit.framework.TestCase2

com.bigdata.btree.raba.codec.AbstractRabaCoderTestCase

All Implemented Interfaces:

junit.framework.Test

Direct Known Subclasses:

AbstractFrontCodedRabaCoderTestCase, TestCanonicalHuffmanRabaCoder, TestConditionalRabaCoder_keys_simple_frontCoded, TestConditionalRabaCoder_values_simple_canonical, TestFixedLengthValueRabaCoder, TestSimpleRabaCoder

public abstract class AbstractRabaCoderTestCase
extends TestCase2

Abstract test suite for IRabaCoder implementations.

Version:

$Id$

Author:

Bryan Thompson

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	AbstractRabaCoderTestCase.TestOp Tests of the Op test helper class.

Nested classes/interfaces inherited from class junit.framework.TestCase2

TestCase2.MyProperties, TestCase2.RandomType

Field Summary

Fields

Modifier and Type	Field and Description
protected IRabaCoder	rabaCoder The fixture under test.
protected static double	scalingFactor Scaling factor converts nanoseconds to milliseconds.

Fields inherited from class junit.framework.TestCase2

_randomType, log

Constructor Summary

Constructors

Constructor and Description
AbstractRabaCoderTestCase()
AbstractRabaCoderTestCase(String name)

Method Summary

Methods

Modifier and Type	Method and Description
protected static void	assertSameIterator(Iterator<byte[]> eitr, Iterator<byte[]> aitr) Verify same byte[] iterators.
static int	doRabaCoderPerformanceTest(IRaba expected, IRabaCoder rabaCoder, int size, int nops, Random r, com.bigdata.btree.raba.codec.AbstractRabaCoderTestCase.Op op) Do a performance stress test consisting of random operations on a randomly generated B+Tree keys IRaba.
protected void	doRandomRoundTripTest(IRabaCoder dataCoder, int size, int capacity) Generates a random byte[][] and verifies round-trip encoding and decoding.
static void	doRoundTripTest(IRabaCoder rabaCoder, IRaba expected)
protected byte[]	getRandomValue(IRabaCoder dataCoder) Return a random byte array.
protected boolean	isFixedLength() Return true if the IRabaCoder only handles fixed length values (the default always returns false).
static void	main(String[] args) Performance stress test for keys.
void	test_empty()
void	test_emptyElement() Test with an empty byte[] element.
void	test_entryCount1() Test with IRaba having a size of ONE (1) and a variety of capacities.
void	test_entryCount2() Test with IRaba having a size of TWO (2) and a variety of capacities.
void	test_error1() This test case was developed for the FrontCodedRabaCoder.
void	test_error2() This test case was developed for the CanonicalHuffmanRabaCoder.
void	test_keyCoderPerformance() A test designed to measure the performance of an IRabaCoder for operations on B+Tree keys, including search.
void	test_mike_personick() A simple unit test.
void	test_negativeByteValues() Test with byte values which are negative values when interpreted as as signed 8 bit integers.
void	test_nsymbolsOne_nulls() Test with a single byte value (nsymbols=1) and some nulls.
void	test_nsymbolsOne() Test with a single byte value (nsymbols:=1).
void	test_randomOnce()
void	test_randomURIs() Test using a sequence of random URIs (formed with successive prefixes).
void	test_withNulls() Test with a null value.
void	test_withNulls2() Test with a single value, which is null.
void	testStress()

Methods inherited from class junit.framework.TestCase2

assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEqualsWithinUlps, assertSameArray, assertSameArray, assertSameBigDecimal, assertSameBigDecimal, assertSameBigInteger, assertSameBigInteger, assertSameIterator, assertSameIterator, assertSameIteratorAnyOrder, assertSameIteratorAnyOrder, assertSameValue, assertSameValue, assertZeroUlps, assertZeroUlps, fail, getInnerCause, getNormalInt, getProjectBuildPath, getProperties, getRandomObject, getRandomObject, getRandomOrder, getRandomString, getTestInputStream, getTestResource, getTestResource, getUlps, getUlps, isDEBUG, isDEBUG, isINFO, isINFO, isInnerCause, logProperties

Methods inherited from class junit.framework.TestCase

assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertFalse, assertFalse, assertNotNull, assertNotNull, assertNotSame, assertNotSame, assertNull, assertNull, assertSame, assertSame, assertTrue, assertTrue, countTestCases, createResult, fail, fail, failNotEquals, failNotSame, failSame, format, getName, run, run, runBare, runTest, setName, setUp, tearDown, toString

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

rabaCoder

protected IRabaCoder rabaCoder

The fixture under test. This will be null unless you explicitly set it in TestCase.setUp().

scalingFactor

protected static final double scalingFactor

Scaling factor converts nanoseconds to milliseconds.

Constructor Detail

AbstractRabaCoderTestCase

public AbstractRabaCoderTestCase()

AbstractRabaCoderTestCase

public AbstractRabaCoderTestCase(String name)

Parameters:

name -

Method Detail

isFixedLength

protected boolean isFixedLength()

Return true if the IRabaCoder only handles fixed length values (the default always returns false). This is used to skip tests which have variable length byte[]s when testing the FixedLengthValueRabaCoder.

test_mike_personick

public void test_mike_personick()
                         throws UnsupportedEncodingException

A simple unit test.

Throws:

UnsupportedEncodingException

test_negativeByteValues

public void test_negativeByteValues()
                             throws UnsupportedEncodingException

Test with byte values which are negative values when interpreted as as signed 8 bit integers.

Throws:

UnsupportedEncodingException

test_emptyElement

public void test_emptyElement()
                       throws UnsupportedEncodingException

Test with an empty byte[] element.

Throws:

UnsupportedEncodingException

test_nsymbolsOne

public void test_nsymbolsOne()

Test with a single byte value (nsymbols:=1). This test was written to a known bug in HuffmanCodec and CanonicalFast64CodeWordDecoder. A workaround for that bug has been implemented in the CanonicalHuffmanRabaCoder.

test_nsymbolsOne_nulls

public void test_nsymbolsOne_nulls()

Test with a single byte value (nsymbols=1) and some nulls. This test was written to a known bug in HuffmanCodec and CanonicalFast64CodeWordDecoder. A workaround for that bug has been implemented in the CanonicalHuffmanRabaCoder.

test_withNulls

public void test_withNulls()
                    throws UnsupportedEncodingException

Test with a null value.

Throws:

UnsupportedEncodingException

test_withNulls2

public void test_withNulls2()

Test with a single value, which is null. FIXME Due to a bug in the CanonicalFast64CodeWordDecoder ctor there is a problem handling a logical byte[][] consisting solely of nulls. We handle this for the case of an empty logical byte[][] using an EmptyRabaValueDecoder. I suppose that could be parameterized to identify the nulls or else just fix the ctor.

test_empty

public void test_empty()
                throws IOException

Throws:

IOException

test_entryCount1

public void test_entryCount1()
                      throws IOException

Test with IRaba having a size of ONE (1) and a variety of capacities.

Throws:

IOException

test_entryCount2

public void test_entryCount2()
                      throws IOException

Test with IRaba having a size of TWO (2) and a variety of capacities.

Throws:

IOException

test_error1

public void test_error1()
                 throws IOException

This test case was developed for the FrontCodedRabaCoder.

Throws:

IOException

test_error2

public void test_error2()

This test case was developed for the CanonicalHuffmanRabaCoder.

 junit.framework.AssertionFailedError: search([44, 186, 169, 175, 191, 31, 36, 227]): expectedIndex=1, actualIndex=-1,
 expected=com.bigdata.btree.raba.ReadOnlyKeysRaba{ capacity=3, size=2, isKeys=true, isReadOnly=true, [
 [44, 186, 169, 175, 191, 31, 36, 12],
 [44, 186, 169, 175, 191, 31, 36, 227]]},
 actual=com.bigdata.btree.raba.codec.CanonicalHuffmanRabaCoder$RabaDecoder{ capacity=2, size=2, isKeys=true, isReadOnly=true, [
 [44, 186, 169, 175, 191, 31, 36, 12],
 [44, 186, 169, 175, 191, 31, 36, 227]]}
     at junit.framework.Assert.fail(Assert.java:47)
     at com.bigdata.btree.AbstractBTreeTestCase.assertSameRaba(AbstractBTreeTestCase.java:583)
     at com.bigdata.btree.raba.codec.AbstractRabaCoderTestCase.doRoundTripTest(AbstractRabaCoderTestCase.java:499)
     at com.bigdata.btree.raba.codec.AbstractRabaCoderTestCase.doRandomRoundTripTest(AbstractRabaCoderTestCase.java:487)
     at com.bigdata.btree.raba.codec.AbstractRabaCoderTestCase.test_entryCount2(AbstractRabaCoderTestCase.java:247)
 

 junit.framework.AssertionFailedError: search([44, 197, 214, 208, 192, 31, 36, 156]): expectedIndex=1, actualIndex=-1,
 expected=com.bigdata.btree.raba.ReadOnlyKeysRaba{ capacity=2, size=2, isKeys=true, isReadOnly=true, [
 [44, 197, 214, 208, 192, 31, 36, 12],
 [44, 197, 214, 208, 192, 31, 36, 156]]},
 actual=com.bigdata.btree.raba.codec.CanonicalHuffmanRabaCoder$RabaDecoder{ capacity=2, size=2, isKeys=true, isReadOnly=true, [
 [44, 197, 214, 208, 192, 31, 36, 12],
 [44, 197, 214, 208, 192, 31, 36, 156]]}
     at junit.framework.Assert.fail(Assert.java:47)
     at com.bigdata.btree.AbstractBTreeTestCase.assertSameRaba(AbstractBTreeTestCase.java:583)
     at com.bigdata.btree.raba.codec.AbstractRabaCoderTestCase.doRoundTripTest(AbstractRabaCoderTestCase.java:538)
     at com.bigdata.btree.raba.codec.AbstractRabaCoderTestCase.test_error2(AbstractRabaCoderTestCase.java:325)
 

test_randomOnce

public void test_randomOnce()
                     throws IOException

Throws:

IOException

testStress

public void testStress()
                throws IOException

Throws:

IOException

test_randomURIs

public void test_randomURIs()
                     throws Exception

Test using a sequence of random URIs (formed with successive prefixes).

Throws:

Exception

getRandomValue

protected byte[] getRandomValue(IRabaCoder dataCoder)

Return a random byte array. The byte array will also have a random length in [0:512] unless the IRabaCoder is a FixedLengthValueRabaCoder, in which case a byte[] having the appropriate length will be returned.

Parameters:

dataCoder - The coder.

Returns:

The random byte[].

doRandomRoundTripTest

protected void doRandomRoundTripTest(IRabaCoder dataCoder,
                         int size,
                         int capacity)
                              throws IOException

Generates a random byte[][] and verifies round-trip encoding and decoding.

Note: This uses a uniform random distribution. Therefore the huffman codes tend to occupy MORE space than the original byte[][] since the canonical huffman code can not be shorter than the original byte values on average when all values are equally likely. When you want to measure the expected compression ratio you need to test with a gaussian distribution (bell curve) over the values, a USASCII distribution, etc.

Parameters:

size - The #of entries in the byte[][].

capacity - The capacity of the byte[][].

Throws:

IOException

doRoundTripTest

public static void doRoundTripTest(IRabaCoder rabaCoder,
                   IRaba expected)

main

public static void main(String[] args)

Performance stress test for keys. Performance tuning should give more weight to coded raba access times, including search and key retrieval or copy, than coding times since most use will be access on the coded data. Those costs are not factored apart in the stress test times. They are parameterized here by a normalized vector of the rates of the different operations (search(), get(), length(), etc).

nops

The #of random operations to be performed. Large values for nops need to be used to get beyond the initial JVM performance tuning so you can more accurately compare the performance of the different coders. For example, a value of 1M (1000000) will run for ~ 30-40s for the front-coded coders. For shorter run times, the order in which we test the coders will dominate their performance.

size

The #of entries in the raba to be tested (must be LTE the capacity)

Parameters:

args - [nops [generator [size]]] FIXME parameterize the generator choice.

test_keyCoderPerformance

public void test_keyCoderPerformance()

A test designed to measure the performance of an IRabaCoder for operations on B+Tree keys, including search.

doRabaCoderPerformanceTest

public static int doRabaCoderPerformanceTest(IRaba expected,
                             IRabaCoder rabaCoder,
                             int size,
                             int nops,
                             Random r,
                             com.bigdata.btree.raba.codec.AbstractRabaCoderTestCase.Op op)

Do a performance stress test consisting of random operations on a randomly generated B+Tree keys IRaba. The operations will be checked against ground truth.

Parameters:

rabaCoder - The coder to be tested.

size - The #of keys.

nops - The #of operations to perform.

r - The random number generator.

op - The distribution of the operations to be performed.

Returns:

The size of the coded record.

assertSameIterator

protected static void assertSameIterator(Iterator<byte[]> eitr,
                      Iterator<byte[]> aitr)

Verify same byte[] iterators.

Parameters:

eitr - The expected iterator.

aitr - The actual iterator.