Michael Thomas Flanagan's Java Scientific Library: Box-Cox transformation

Michael Thomas Flanagan's Java Scientific Library

BoxCox Class: Box-Cox transformation

Last update: 7 December 2011 Main Page of Michael Thomas Flanagan's Java Scientific Library

This class contains methods for performing the Box-Cox transformation

on an array of data, y_i. The Box-Cox transformation is used to attempt to transform an array of data, y_i, to one, y_i(λ), that conforms to a sample taken from a Gaussian (normal) distribution.

In this class, the data, y_i, is first standardized:

where

and s are the mean and standard deviation of the y_i.
The Box-Cox parameter, λ₂, is set at:

An optimum value of the Box-Cox parameter, λ₁, is calculated as the value of λ₁ maximizing the correlation coefficient of a Gaussian (normal) probability plot of the transformed data. This maximization is performed using the Maximisation class which employs a Nelder and Mead Simplex optimization.
Probabilty plot, skewness and kurtosis merthods are present to facilitate checking the efficacy of the transformation.
An inverse transform method and a fixed value (λ₁ and λ₂) method are also available.

See Stat class for the general statistical methods and functions.

import directive: import flanagan.analysis.BoxCox;

Table of methods
Listing of the source code (Last update: 7 December 2011)
Other classes, in this library, used by this class
Main Page of Michael Thomas Flanagan's Java Scientific Library

SUMMARY OF METHODS

Constructors			public BoxCox(double[] ydata)
			public BoxCox(float[] ydata)
			public BoxCox(int[] ydata)
			public BoxCox(long[] ydata)
			public BoxCox(short[] ydata)
			public BoxCox(byte[] ydata)
			public BoxCox(BigDecimal[] ydata)
			public BoxCox(BigInteger[] ydata)
			public BoxCox(ArrayMaths ydata)
			public BoxCox(ArrayList<Object> ydata)
			public BoxCox(Vector<Object> ydata)
			public BoxCox(Stat ydata)
Analysis	Box-Cox transform and full anlysis		public void analysis()
			public void analysis(String fileTitle)
Transformed Data	Transform data with best estimates of λ₁ and λ₂		public double[] transform()
	Scaled transformed data		public double[] scaledTransformedData()
	Ordered scaled transformed data		public double[] orderedScaledTransformedData()
	Standardized transformed data		public double[] standardizedTransformedData()
	Transformed data		public double[] transformedData()
	λ₁		public double lambdaOne()
	λ₂		public double lambdaTwo()
	Gaussian Probability Plot	Display plot	public void transformedProbabiltyPlot()
		Correlation coefficient	public double transformedCorrelationCoefficient()
		Gradient	public double transformedGradient()
			public double transformedGradientError()
		Intercept	public double transformedIntercept()
			public double transformedInterceptError()

	Mean		public double transformedMean()
	Standard deviation		public double transformedStandardDeviation()
	Standard error of the mean		public double transformedStandardError()
	Moment skewness		public double transformedMomentSkewness()
	Median skewness		public double transformedMedianSkewness()
	Quartile skewness		public double transformedQuartileSkewness()
	Excess kurtosis		public double transformedExcessKurtosis()
	Median		public double transformedMedian()
	Minimum		public double transformedMinimum()
	Maximum		public double transformedMaximum()
Range		public double transformedRange()
Fixed Value Transform	Transform data with user supplied values of λ₁ and λ₂		public double[] fixedValueTransform(double lambdaOne, double lambdaTwo)
			public double[] fixedValueTransform(double lambdaOne)
Inverse Transform	Perform inverse transform		public double[] inverseTransform(double lambdaOne, double lambdaTwo)
			public double[] inverseTransform(double lambdaOne)
	Shift factor		public double lambdaThree()
Original Data	Original data		public double[] originalData()
	standardized original data		public double[] standardizedOriginalData()
	Sorted original data		public double[] sortedOriginalData()
	Shifted standardized original data		public double[] shiftedStandardizedOriginalData()
	Gaussian Probability Plot	Display plot	public void originalProbabiltyPlot()
		Correlation coefficient	public double originalCorrelationCoefficient()
		Gradient	public double originalGradient()
			public double originalGradientError()
		Intercept	public double originalIntercept()
			public double originalInterceptError()
	Mean		public double originalMean()
	Standard deviation		public double originalStandardDeviation()
	Standard error of the mean		public double originalStandardError()
	Moment skewness		public double originalMomentSkewness()
	Median skewness		public double originalMedianSkewness()
	Quartile skewness		public double originalQuartileSkewness()
	Excess kurtosis		public double originalExcessKurtosis()
	Median		public double originalMedian()
	Minimum		public double originalMinimum()
	Maximum		public double originalMaximum()
	Range		public double originalRange()
Set the variance denominator	Set denominator to n		public void setDenominatorToN()

CONSTRUCTORS

public BoxCox(double[] ydata)
public BoxCox(float[] ydata)
public BoxCox(int[] ydata)
public BoxCox(long[] ydata)
public BoxCox(short[] ydata)
public BoxCox(byte[] ydata)
public BoxCox(BigDecimal[] ydata)
public BoxCox(BigInteger[] ydata)
public BoxCox(ArrayMaths ydata)
public BoxCox(ArrayList<Object> ydata)
public BoxCox(Vector<Object> ydata)
public BoxCox(Stat ydata)
Usage: BoxCox bc = new BoxCox(ydata);
Creates an instance of BoxCox for transformation of the data entered as the argument ydata. The data are stored as an array of double but may be entered as:

a one dimensional array of double, float, long, int, short, byte, BigDecimal or BigInteger
an ArrayMaths object, an ArrayList<Object> or a Vector<Object>
the internal array of an instance of Stat enterted as the instance of Stat

PERFORM TRANSFORM AND TRANSFORMED DATA

Perform Box-Cox transform
public double[] transform()
Usage: data = bc.transform();
This method performs the Box-Cox transformation

on an array of data, y_i, entered via the constructor argument. In this method, the data, y_i, is first standardized:

where

and s are the mean and standard deviation of the y_i.
The Box-Cox parameter, λ₂, is set at:

An optimum value of the Box-Cox parameter, λ₁, used in this transformation, is calculated as the value of λ₁ maximizing the correlation coefficient of a Gaussian (normal) probability plot of the transformed data. This maximization is performed using the Maximisation class which employs a Nelder and Mead Simplex optimization.
This method returns the transformed data scaled to the mean and standard deviation of the original data.

Scaled transformed data
public double[] scaledTransformedData()
Usage:                      data = bc.scaledTransformedData();
This method returns the transformed data scaled to the mean and standard deviation of the original data.

Ordered scaled transformed data
public double[] orderedScaledTransformedData()
Usage:                      data = bc.orderedScaledTransformedData();
This method returns the transformed data scaled to the mean and standard deviation of the original data and ordered into ascending values.

Standardized transformed data
public double[] standardizedTransformedData()
Usage:                      data = bc.standardizedTransformedData();
This method returns the standardized transformed data.

Transformed data
public double[] transformedData()
Usage:                      transformedData = bc.transformedData();
This method returns the transformed data obtained on transforming the shifted standardized original data using the values of λ₁ and λ₁ that may be obtained using the methods below.

Box-Cox parameter, λ₁
public double lambdaOne()
Usage:                      transformedData = bc.lambdaOne();
This method returns the Box-Cox parameter, λ₁, giving a maximum value of the probabilty plot correlation coefficient. See above for more details.
If the value of λ₁ has been entered, i.e. via a fixed value or inverse transform, this entered value is returned by this method.

Box-Cox parameter, λ₂
public double lambdaTwo()
Usage:                      transformedData = bc.lambdaTwo();
This method returns the Box-Cox parameter, λ₂, set as

See above for more details.
If the value of λ₂ has been entered, i.e. via a fixed value or inverse transform, this entered value is returned by this method.

Gaussian Probability Plot

Display Gaussian probability plot
public void transformedProbabiltyPlot()
Usage: bc.transformedProbabiltyPlot();
This method displays a plot of a Gaussian (normal) probability plot for the transformed data. The plot legend also displays the values of λ₁, λ₂, the correlation coefficient and the intercept and gradient of the best fit to a straight line.
Correlation coefficient of the Gaussian probability plot
public double transformedCorrelationCoefficient()
Usage: corrCoeff = bc.transformedCorrelationCoefficient();
This method returns the correlation coefficient of the Gaussian (normal) probability plot of the transformed data.
Gradient of the Gaussian probability plot
public double transformedGradient()
Usage: gradient = bc.transformedGradient();
This method returns the gradient of the best fit straight line to the Gaussian (normal) probability plot of the transformed data.
Estimated Error of the Gradient of the Gaussian probability plot
public double transformedGradientError()
Usage: gradientError = bc.transformedGradientError();
This method returns the estimate of the error of the gradient of the best fit straight line to the Gaussian (normal) probability plot of the transformed data.
Intercept of the Gaussian probability plot
public double transformedIntercept()
Usage: intercept = bc.transformedIntercept();
This method returns the intercept of the best fit straight line to the Gaussian (normal) probability plot of the transformed data.
Estimated Error of the Intercept of the Gaussian probability plot
public double transformedInterceptError()
Usage: interceptError = bc.transformedInterceptError();
This method returns the estimate of the error of the intercept of the best fit straight line to the Gaussian (normal) probability plot of the transformed data.

Mean
public double transformedMean()
Usage:                      mean = bc.transformedMean();
This method returns the mean of the scaled transformed data.

Standard deviation
public double transformedStandardDeviation()
Usage:                      sd = bc.transformedStandardDeviation();
This method returns the standard deviation of the scaled transformed data.

Standard error of the mean
public double transformedStandardError()
Usage:                      sd = bc.transformedStandardError();
This method returns the standard error of the mean of the scaled transformed data.

Moment skewness
public double transformedMomentSkewness()
Usage:                      skewness = bc.transformedMomentSkewness();
This method returns the Moment skewness of the scaled transformed data.

Median skewness
public double transformedMedianSkewness()
Usage:                      skewness = bc.transformedMedianSkewness();
This method returns the Median skewness of the scaled transformed data.

Quartile skewness
public double transformedQuartileSkewness()
Usage:                      skewness = bc.transformedQuartileSkewness();
This method returns the Quartile skewness of the scaled transformed data.

Excess kurtosis
public double transformedExcessKurtosis()
Usage:                      kurtosis = bc.transformedExcessKurtosis();
This method returns the Excess kurtosis of the scaled transformed data.

Median
public double transformedMedian()
Usage:                      median = bc.transformedMedian();
This method returns the median value of the scaled transformed data.

Minimum
public double transformedMinimum()
Usage:                      minimum = bc.transformedMinimum();
This method returns the minimum value of the scaled transformed data.

Maximum
public double transformedMaximum()
Usage:                      maximum = bc.transformedMaximum();
This method returns the maximum value of the scaled transformed data.

Range
public double transformedRange()
Usage:                      range = bc.transformedRange();
This method returns the range of the scaled transformed data.

PERFORM FIXED VALUE TRANSFORM

Perform Box-Cox transform with user supplied values of λ₁ and λ₂
public double[] fixedValueTransform(double lambdaOne, double lambdaTwo)
public double[] fixedValueTransform(double lambdaOne)
Usage: data = bc.fixedValueTransform(lambdaOne, lambdaTwo);
This method performs the Box-Cox transformation

on an array of data, y_i, entered via the constructor argument. In this method, the data, y_i, is first standardized:

where

and s are the mean and standard deviation of the y_i.
The Box-Cox parameters λ₁ and λ₂ used are those supplied via the arguments of this method, lambdaOne and lambdaTwo. Obviously this method will not necessarily return a data set that may conform to a sample taken from a Gaussian (normal) distribution. See above for a method that attempts to achieve a full Box-Cox transformation.
This method returns the transformed data scaled to the mean and standard deviation of the original data.

Usage: data = bc.fixedValueTransform(lambdaOne);
This method performs the Box-Cox transformation

on an array of data, y_i, entered via the constructor argument. In this method, the data, y_i, is first standardized:

where

and s are the mean and standard deviation of the y_i.
The Box-Cox parameter λ₁ used is that supplied via the argument of this method, lambdaOne, i.e. lambdaTwo is set to zero. Obviously this method will not necessarily return a data set that may conform to a sample taken from a Gaussian (normal) distribution. See above for a method that attempts to achieve a full Box-Cox transformation.
This method returns the transformed data scaled to the mean and standard deviation of the original data.

INVERSE TRANSFORM

Perform Inverse Transform
public double[] inverseTransform(double lambdaOne, double lambdaTwo)
public double[] inverseTransform(double lambdaOne)
Usage: inverse = bc.inverseTransform(lambdaOne, lambdaTwo);
This method returns the inverse of the Box Cox transform:

The array entered via a constructor, y, is inverse transformed to the array z and returned to inverse in the above usage. The parameter, λ₃, is returned as zero unless the data leads to imaginary transformed data. If this were to occur the original data is shifted so that all values lead to real transformed values. The method for getting the shift factor, λ₃, is described below.
The parameters λ₁ and λ₂ have the same meaning as described above for the Box Cox transform.

Usage: inverse = bc.inverseTransform(lambdaOne);
This method returns the inverse of the Box Cox transform:

The array entered via a constructor, y, is inverse transformed to the array z and returned to inverse in the above usage. The parameter, λ₃, is returned as zero unless the data leads to imaginary transformed data. If this were to occur the original data is shifted so that all values lead to real transformed values. The method for getting the shift factor, λ₃, is described below.
The parameters λ₁ has the same meaning as described above for the Box Cox transform and λ₂ is set to zero.

Return the value of the inverse transform shift factor
public double lambdaThree()
Usage: lambda3 = bc.lambdaThree();
Returns the shift parameter, λ₃, as described in the Inverse Transform section immediately above.

ORIGINAL DATA

Original data
public double[] originalData()
Usage:                      data = bc.originalData();
This method returns the original data as an array of doubles.

Standardized original data
public double[] standardizedOriginalData()
Usage:                      data = bc.standardizedOriginalData();
This method returns the original data standardized to a mean of zero and a standard deviation of unity.

Sorted original data
public double[] sortedOriginalData()
Usage:                      data = bc.sortedOriginalData();
This method returns the original data ordered into ascending values.

Shifted standardized original data
public double[] shiftedStandardizedOriginalData()
Usage:                      data = bc.shiftedStandardizedOriginal();
This method returns the original data after standarization and a shift of λ₂. This is the data array that is transformed.

Gaussian Probability Plot

Display Gaussian probability plot
public void originalProbabiltyPlot()
Usage: bc.originalProbabiltyPlot();
This method displays a plot of a Gaussian (normal) probability plot for the original data. The plot legend also displays the values of λ₁, λ₂, the correlation coefficient and the intercept and gradient of the best fit to a straight line.
Correlation coefficient of the Gaussian probability plot
public double originalCorrelationCoefficient()
Usage: corrCoeff = bc.originalCorrelationCoefficient();
This method returns the correlation coefficient of the Gaussian (normal) probability plot of the original data.
Gradient of the Gaussian probability plot
public double originalGradient()
Usage: gradient = bc.originalGradient();
This method returns the gradient of the best fit straight line to the Gaussian (normal) probability plot of the original data.
Estimated Error of the Gradient of the Gaussian probability plot
public double originalGradientError()
Usage: gradientError = bc.originalGradientError();
This method returns the best estimate of the error of the gradient of the best fit straight line to the Gaussian (normal) probability plot of the original data.
Intercept of the Gaussian probability plot
public double originalIntercept()
Usage: intercept = bc.originalIntercept();
This method returns the intercept of the best fit straight line to the Gaussian (normal) probability plot of the original data.
Estimated Error of the Intercept of the Gaussian probability plot
public double originalGradientError()
Usage: interceptError = bc.originalInterceptError();
This method returns the best estimate of the error of the intercept of the best fit straight line to the Gaussian (normal) probability plot of the original data.

Mean
public double originalMean()
Usage:                      mean = bc.originalMean();
This method returns the mean of the scaled original data.

Standard deviation
public double originalStandardDeviation()
Usage:                      sd = bc.originalStandardDeviation();
This method returns the standard deviation of the scaled original data.

Standard error of the mean
public double originalStandardError()
Usage:                      sd = bc.originalStandardError();
This method returns the standard error of the mean of the scaled original data.

Moment skewness
public double originalMomentSkewness()
Usage:                      skewness = bc.originalMomentSkewness();
This method returns the Moment skewness of the scaled original data.

Median skewness
public double originalMedianSkewness()
Usage:                      skewness = bc.originalMedianSkewness();
This method returns the Median skewness of the scaled original data.

Quartile skewness
public double originalQuartileSkewness()
Usage:                      skewness = bc.originalQuartileSkewness();
This method returns the Quartile skewness of the scaled original data.

Excess kurtosis
public double originalExcessKurtosis()
Usage:                      kurtosis = bc.originalExcessKurtosis();
This method returns the Excess kurtosis of the scaled original data.

Median
public double originalMedian()
Usage:                      median = bc.originalMedian();
This method returns the median value of the scaled original data.

Minimum
public double originalMinimum()
Usage:                      minimum = bc.originalMinimum();
This method returns the minimum value of the scaled original data.

Maximum
public double originalMaximum()
Usage:                      maximum = bc.originalMaximum();
This method returns the maximum value of the scaled original data.

Range
public double originalRange()
Usage:                      range = bc.originalRange();
This method returns the range of the scaled original data.

ANALYSIS

public void analysis(String fileTitle)
public void analysis()
Usage: bc.analysis(fileTitle);
This method displays:

a Gaussian (normal) probability plot of the transformed data
a Gaussian (normal) probability plot of the original data

and writes to a text file, fileTitle:

the time and date ofthe program's execution
the Box-Cox parameter λ₁
the Box-Cox parameter λ₂

and for both the transformed data, scaled to the original mean and standard deviation, and for the original data:

the probability plot correlation coefficient
the probability plot gradient and its estimated error
the probability plot intercept and its estimated error
the mean of the data
the median of the data
the standard deviation of the data
the standard error of the mean of the data
the moment skewness of the data
the median skewness of the data
the quartile skewness of the data
the excess kurtosis of the data
the minimum of the data
the maximum of the data
the range of the data

Usage: bc.analysis();
This method is identical to analysis(fileTitle) above, except that in the absence of a supplied file title the output file is named BoxCoxAnalysis.txt.

SET VARIANCE DENOMINATOR TO N

public void setDenominatorToN()
Usage: bc.setDenominatorToN();
This method sets the denominator in calculations of variance, standard deviation, skewness and kurtosis to the number of data points, n. The default value is n−1. See Stat class for details.

OTHER CLASSES USED BY THIS CLASS

This class uses the following classes in this library:

This page was prepared by Dr Michael Thomas Flanagan