Data Generation

Introduction¶

This notebook demonstrates the data mining package written in Maxima, which is helpful for

• computing some mathematical criteria precisely without numerical error/instability, and
• creating randomized moodle stack questions.

The implementations are simplified and may not be scalable to large data sets.

To load the package, run the following cell:

To learn Maxima, you may use the describe function or refer to the documentation for more details:

As an example, the following defines a function that computes the maxima of its arguments:

In the above example, maxima([lst]) is a recursive function that

• takes a variable number of arguments, which will be stored in lst as a list, and
• returns a list [i,m] as follows:
  • If lst is non-empty, lst[i]=m is a maximum element of lst and i is the smallest such index.
  • If lst is empty, then [0,-inf] is returned, following the conventions that
    • the maximum element of an empty list [] of numbers is -inf, and
    • Maxima uses 1-based numbering so 0 is the index of an imaginary item before the first item in a list.

Generate data from lists¶

Data is a matrix of feature values associated with feature names. Data can be created by build_data_from_list(fns, lst) where

• fns is a list of feature names, and
• lst is a list of instances, which are lists of feature values corresponding to the feature names.

Information of the data can be obtained using other functions:

• feature_names(data) returns the feature names of data.
• size(data) returns the number of instances of data.
• feature_index(fns, fn) returns the index of a feature named fn in the list fns of feature names.
• get_data(data, i) returns the i-th instance of data.
• feature_values(data, fn) returns the list of feature values of the feature fn.
• plot_labeled_data(data,xy,target)

plot_labeled_data(data,xy,target)

plots the labeled data where

• xy specifies the pair of features for the $x$ and $y$ axes, and
• target is used to color code the data points.

Generate data with rules¶

Data can also be generated (randomly) according to some specified rules using build_data(fns, gen, n) where

• fns is a list of feature names,
• gen is a function that takes a unique index and generates an instance associated with the index, and
• n is the number of instances to generate.

In the above example,

• $i$ is the unique index,
• $X_1$ and $X_2$ are uniformly random generated from $\Set{0,1,2}$, and
• $Y$ is a deterministic function of $X_1$ and $X_2$, namely,
  
  $$Y=\begin{cases} 1 & X_1<1, X_2>0\\ 0 & \text{otherwise.} \end{cases}$$

  (1)

Transform features¶

New features can be created by transforming existing ones using transform_features(data, nfns, ngen) where

• data is a data set,
• nfns is the list of new feature names, and
• ngen is a function that takes a unique index and returns an instance.

In the above example,

• the features $X1$ and $X2$ in data are transformed to create the feature $Y$, and
• the row index is used to create the feature $i$.

Subsample data¶

To subsample data based on specific conditions, use subsample_data(data, cond) where

• data is the data to subsample, and
• cond is a function that takes a row index and returns a boolean expression on the feature names.

It returns data but keeping only the instances indexed by i where cond[i] evaluates to true with the feature names substituted by the corresponding feature values.

In the above example, only instances with $X_1<1$ and $X_2>0$ are returned.

Combine data¶

Data can be stacked (vertically) by stack_data(data_1, data_2, ...) where data_i’s are data with the same list of features.

In the above example, data consists of instances from data_1 and data_2.