\chapter{Modeling (\module{cvxopt.modeling})}
\label{chap:modeling}
The module \module{cvxopt.modeling} can be used to specify and solve 
optimization problems  with convex piecewise-linear objective and 
constraint functions.

To specify an optimization problem one first defines 
the optimization variables (see section~\ref{s-variables}),
and then defines the objective and constraint functions 
using linear operations (vector addition and subtraction,
matrix-vector multiplication, indexing and slicing)
and nested evaluations of \function{max()}, \function{min()}, 
\function{abs()} and \function{sum()} (see section~\ref{s-functions}).

\section{Variables} \label{s-variables}
Optimization variables are represented by \pytype{variable} objects.

\begin{classdesc}{variable}{\optional{size\optional{, name}}}
A vector variable.  The first argument is the dimension of the
vector (a positive integer with default value 1). 
The second argument is a string with a name for the variable.
The name is optional and has default value \code{""}. It is only used 
when displaying variables (or objects that depend on variables, such 
as functions or constraints) 
using \function{print} statements, when calling the built-in functions
\function{repr()}  or \function{str()}, or when writing linear programs
to MPS files.
\end{classdesc}
The function \function{len()} returns the length of a 
\pytype{variable}.  A \pytype{variable} \var{x} has two attributes.
\begin{memberdesc}{name}
The name of the variable.  
\end{memberdesc}

\begin{memberdesc}{value}
Either \None\  or a dense \dtc\ matrix of size \code{len(\var{x})} by 1.

The attribute \var{x}.\member{value} is set to \None\ when the 
variable \var{x} is created.   It can be given a numerical value later, 
typically by solving an LP that has \var{x} as one of its variables.   
One can also make an explicit assignment 
\code{\var{x}.\member{value} = \var{y}}.  The
assigned value \var{y} must be an \intgr\ or \flt, or a 
dense \dtc\ matrix of size (\code{len(\var{x})},1). 
If \var{y} is an \intgr\ or \flt\, all the elements of 
\var{x}.\member{value} are set to the value of \var{y}.
\end{memberdesc}

\begin{verbatim}
>>> from cvxopt.base import matrix
>>> from cvxopt.modeling import variable
>>> x = variable(3,'a')
>>> len(x)
3
>>> print x.name
a
>>> print x.value 
None
>>> x.value = matrix([1.,2.,3.])
>>> print x.value
[ 1.00e+00]
[ 2.00e+00]
[ 3.00e+00]
>>> x.value = 1
>>> print x.value
[ 1.00e+00]
[ 1.00e+00]
[ 1.00e+00]
\end{verbatim}


\section{Functions} \label{s-functions}
Objective and constraint functions can be defined via overloaded 
operations on variables and other functions.  A function \var{f} is 
interpreted as a column vector, with length \code{len(\var{f})} 
and with a value that depends on the values of its variables.  
Functions have two public attributes.  
\begin{methoddesc}{variables}
Returns a copy of the list of variables of the function.
\end{methoddesc}

\begin{methoddesc}{value}
The function value.  If any of the variables of \var{f} has value 
\None, then \var{f}.\function{value()} returns \None.
Otherwise, it returns a dense \dtc\ matrix of size 
(\code{len(\var{f}),1}) with the function value computed from the 
\member{value} attributes of the variables of \var{f}.  
\end{methoddesc}

Three types of functions are supported: affine, convex 
piecewise-linear and concave piecewise-linear.

\textbf{Affine functions} represent vector valued functions of the form
\[
 f(x_1,\ldots,x_n) = A_1 x_1 + \cdots + A_n x_n + b.
\]
The coefficients can be scalars or dense or sparse matrices. The 
constant term is a scalar or a column vector.

Affine functions result from the following operations.
\begin{description}
\item[Unary operations]
For a variable \var{x}, the unary operation \code{+\var{x}} results in 
an affine function with \var{x} as variable, coefficient 1.0, and 
constant term 0.0.  The unary operation \code{-\var{x}} returns an 
affine function with \var{x} as variable, coefficient -1.0, and 
constant term 0.0.  For an affine function \var{f}, \code{+\var{f}} is 
a copy of \var{f}, and \code{-\var{f}} is a copy of \var{f} with the 
signs of its coefficients and constant term reversed.

\item[Addition and subtraction]
Sums and differences of affine functions, variables and constants 
result in new affine functions.
The constant terms in the sum can be of type \intgr\ or \flt, or
dense or sparse \dtc\ matrices with one column. 

The rules for addition and subtraction follow the conventions for 
matrix addition and subtraction in sections~\ref{s-arithmetic}
and~\ref{s-spmatrix-arith}, with variables and affine functions 
interpreted as dense \dtc\ matrices with one column.
In particular, a scalar term (\intgr, \flt, 1 by 1 dense \dtc\ matrix, 
variable of length 1, or affine function of length 1) can be added 
to an affine function or variable of length greater than 1.

\item[Multiplication]
Suppose \var{v} is an affine function or a variable, and \var{a} is 
an \intgr, \flt, sparse or dense \dtc\ matrix.  The products 
\code{\var{a}*\var{v}} and  \code{\var{v}*\var{a}} are 
valid affine functions whenever the product is allowed under the rules 
for matrix and scalar multiplication  of sections~\ref{s-arithmetic} 
and~\ref{s-spmatrix-arith}, with \var{v} interpreted as a \dtc\ matrix 
with one column.
In particular, the product \code{\var{a}*\var{v}} is defined if 
\var{a} is a scalar (\intgr, \flt\ or 1 by 1 dense \dtc\ matrix), 
or a matrix (dense or sparse) with 
\code{\var{a}.\member{size}[1] = len(\var{v})}.   
The operation \code{\var{v}*\var{a}} is defined if \var{a} is scalar,
or if \code{len(\var{v})} = 1 and \var{a} is a matrix with one 
column.

\item[Inner products]
The following two functions return scalar affine functions defined
as inner products of a constant vector with  a variable or affine
function.
\begin{funcdesc}{sum}{v}
The argument is an affine function or a variable.  The result is an
affine function of length 1, with the sum of the components of the
argument \var{v}.  
\end{funcdesc}

\begin{funcdesc}{dot}{u,v}
If \var{v} is a variable or affine function and \var{u} is a \dtc\
matrix of size  (\code{len(\var{v})},1), then 
\code{dot(\var{u},\var{v})} and \code{dot(\var{v},\var{u})} are 
equivalent to \code{\var{u}.\function{trans()}*\var{v}}.

If \var{u} and \var{v} are dense matrices, then 
\code{dot(\var{u},\var{v})} 
is equivalent to the function \code{blas.dot(\var{u},\var{v})} 
defined in section~\ref{s-blas1}, \ie, it returns the inner product of 
the two matrices.
\end{funcdesc}
\end{description}

In the following example, the variable \var{x} has length 1 and
\var{y} has length 2.
The functions \var{f} and \var{g} are given by
\BEAS
 f(x,y) & = & \left[ \begin{array}{c} 2 \\ 2 \end{array}\right] x 
  + y + \left[ \begin{array}{c} 3 \\ 3 \end{array}\right], \\
 g(x,y) & = &
  \left[ \begin{array}{cc} 1 & 3 \\ 2 & 4 \end{array}\right] f(x,y)
  + \left[ \begin{array}{cc} 1 & 1 \\ 1 & 1 \end{array} \right] y + 
  \left[ \begin{array}{c} 1 \\ -1 \end{array} \right] \\
 & = & \left[ \begin{array}{c} 8 \\ 12 \end{array}\right] x 
  + \left[ \begin{array}{cc} 2 & 4 \\ 3 & 5 \end{array}\right] y
  + \left[ \begin{array}{c} 13 \\ 17\end{array}\right].
\EEAS
\begin{verbatim}
>>> from cvxopt.modeling import variable
>>> x = variable(1,'x')
>>> y = variable(2,'y')
>>> f = 2*x + y + 3  
>>> A = matrix([[1., 2.], [3.,4.]])
>>> b = matrix([1.,-1.])
>>> g = A*f + sum(y) + b 
>>> print g
affine function of length 2
constant term:
[ 1.30e+01]
[ 1.70e+01]
linear term: linear function of length 2
coefficient of variable(2,'y'):
[ 2.00e+00  4.00e+00]
[ 3.00e+00  5.00e+00]
coefficient of variable(1,'x'):
[ 8.00e+00]
[ 1.20e+01]
\end{verbatim}

\begin{description}
\item[In-place operations] 
For an affine function \var{f} the operations \code{\var{f} += \var{u}} 
and \code{\var{f} -= \var{u}}, with \var{u} a constant, a variable or 
an affine function, are allowed if they do not change the length of 
\var{f}, \ie, if \var{u} has length \code{len(\var{f})} or length 1.
In-place multiplication \code{\var{f} *= \var{u}} and division 
\code{\var{f} /= \var{u}} are allowed if \var{u} is an \intgr, \flt, or 
1 by 1 matrix.
\end{description}

\begin{description}
\item[Indexing and slicing] Variables and affine functions admit 
single-argument indexing of the four types described in 
section~\ref{s-indexing}.  The result of an indexing or slicing 
operation is an affine function.  
\end{description}

\begin{verbatim}
>>> x = variable(4,'x')
>>> f = x[::2]
>>> print f 
linear function of length 2
linear term: linear function of length 2
coefficient of variable(4,'x'):
[ 1.00e+00     0         0         0    ]
[    0         0      1.00e+00     0    ]
>>> y = variable(3,'x')
>>> g = matrix(range(12),(3,4),'d')*x - 3*y + 1
>>> print g[0] + g[2]
affine function of length 1
constant term:
[ 2.00e+00]
linear term: linear function of length 1
coefficient of variable(4,'x'):
[ 2.00e+00  8.00e+00  1.40e+01  2.00e+01]
coefficient of variable(3,'x'):
[-3.00e+00     0     -3.00e+00]
\end{verbatim}


The general expression of a \textbf{convex piecewise-linear} function is
\[
 f(x_1,\ldots,x_n) = b + A_1 x_1 + \cdots + A_n x_n + 
  \sum_{k=1}^K \max (y_1, y_2, \ldots, y_{m_k}).
\]
The maximum in this expression is a componentwise maximum of its 
vector arguments, which can be constant vectors, variables, affine 
functions or convex piecewise-linear functions.
The general expression for a \textbf{concave piecewise-linear} function
is
\[
 f(x_1,\ldots,x_n) = b + A_1 x_1 + \cdots + A_n x_n + 
  \sum_{k=1}^K \min (y_1, y_2, \ldots, y_{m_k}).
\]
Here the arguments of the \function{min()} can be constants, variables, 
affine functions or concave piecewise-linear functions.

Piecewise-linear functions can be created using the following 
operations.
\begin{description}
\item[Maximum]  If the arguments in 
\code{\var{f} = max(\var{y1},\var{y2}, \ldots)} 
do not include any variables or functions, then the Python built-in 
\function{max()} is evaluated.  

If one or more of the arguments are variables or functions, 
\function{max()} returns a piecewise-linear function defined as the 
elementwise maximum of its arguments. 
In other words, \code{\var{f}[\var{k}] = 
max(\var{y1}[\var{k}],\var{y2}[\var{k}], \ldots)} 
for \var{k}=0, \ldots, \code{len(\var{f})-1}.
The length of \var{f} is equal to the maximum of the lengths of the 
arguments.  Each argument must have length equal to 
\code{len(\var{f})} or length one.  
Arguments with length one are interpreted as vectors of 
length \code{len(\var{f})} with identical entries.

The arguments can be scalars of type \intgr\ or \flt, 
dense \dtc\ matrices with one column, variables, affine functions or 
convex piecewise-linear functions.
 
With one argument, \code{\var{f} = max(\var{u})} is interpreted as
\code{\var{f} = max(\var{u}[0],\var{u}[1],\ldots,
\var{u}[\function{len}(\var{u})-1])}.  

\item[Minimum] Similar to \function{max()} but returns a concave 
piecewise-linear function.  The arguments can be scalars of type 
\intgr\ or \flt, dense \dtc\ matrices with one column, 
variables, affine functions or concave piecewise-linear functions.

\item[Absolute value] 
If \var{u} is a variable or affine function then 
\code{\var{f} = abs(\var{u})} returns the convex piecewise-linear 
function \code{max(\var{u},-\var{u})}.

\item[Unary plus and minus] \code{+\var{f}}\ creates a copy of \var{f}.
\code{-\var{f}}\ is a concave piecewise-linear function if \var{f} is 
convex and a convex piecewise-linear function if \var{f} is concave.

\item[Addition and subtraction]  Sums and differences involving 
piecewise-linear functions are allowed if they result in convex
or concave functions. For example, one can add two convex or two
concave functions, but not a convex and a concave function.
The command \code{sum(\var{f})} is equivalent
to \code{\var{f}[0] + \var{f}[1] + \ldots + \var{f}[len(\var{f})-1]}.

\item[Multiplication] Scalar multiplication \code{\var{a}*\var{f}} of a 
piecewise-linear function \var{f} is defined if \var{a}
is an \intgr, \flt, 1 by 1 \dtc\ matrix. 
Matrix-matrix multiplications \code{\var{a}*\var{f}} or 
\code{\var{f}*\var{a}} are only defined if \var{a} is a dense or 
sparse 1 by 1 matrix.

\item[Indexing and slicing] Piecewise-linear functions admit 
single-argument indexing of the four types described in 
section~\ref{s-indexing}.  The result of an indexing or slicing 
operation is a new piecewise-linear function.
\end{description}

In the following example, \var{f} is the 1-norm of a vector
variable \var{x} of length 10, \var{g} is its infinity-norm and 
\var{h} is the function
\[
  h(x) = \sum_k  \phi(x[k]),\qquad
  \phi(u) = \left\{\begin{array}{ll}
     0 & |u| \leq 1 \\
    |u|-1  & 1 \leq |u| \leq 2 \\
    2|u|-3  & |u| \geq 2. \end{array}\right.
\]
\begin{verbatim}
>>> from cvxopt.modeling import variable, max
>>> x = variable(10, 'x')
>>> f = sum(abs(x))    
>>> g = max(abs(x))   
>>> h = sum(max(0, abs(x)-1, 2*abs(x)-3))  
\end{verbatim}

\begin{description}
\item[In-place operations]
If \var{f} is piecewise-linear then the in-place operations  
\code{\var{f} += \var{u}}, \code{\var{f} -= \var{u}}, 
\code{\var{f} *= \var{u}}, \code{\var{f} /= \var{u}} are defined if the 
corresponding expanded operations \code{\var{f} = \var{f}+\var{u}}, 
\code{\var{f} = \var{f}-\var{u}}, \code{\var{f} = \var{f}*\var{u}}
and \code{\var{f} = \var{f}/\var{u}} are defined and if they do not 
change the length of \var{f}.
\end{description}

\section{Constraints}
Linear equality and inequality constraints of the form
\[
  f(x_1,\ldots,x_n) = 0, \qquad f(x_1,\ldots,x_n) \preceq  0, 
\]
where \tm{f} is a convex function, are represented by \pytype{constraint}
objects.  Equality constraints are created by expressions of the form 
\begin{quote}
\code{\var{f1} == \var{f2}}. 
\end{quote}
Here \var{f1} and \var{f2} can be any objects for which the 
difference \code{\var{f1}-\var{f2}} yields an affine function.  
Inequality constraints are created by expressions of the form 
\begin{quote}
\code{\var{f1} <= \var{f2}}, \qquad \code{\var{f2} >= \var{f1}},
\end{quote}
where \var{f1} and \var{f2} can be any objects for which the difference 
\code{\var{f1}-\var{f2}} yields a convex piecewise-linear function.  
The comparison operators first convert the expressions to 
\code{\var{f1}-\var{f2} == 0}, resp.\, \code{\var{f1}-\var{f2} <= 0},
and then return a new \pytype{constraint}\ object with constraint 
function
\code{\var{f1}-\var{f2}}.

In the following example we create three constraints
\[
  0 \preceq x \preceq \ones, \qquad \ones^T x = 2,
\]
for a variable of length 5.
\begin{verbatim}
>>> x = variable(5,'x')
>>> c1 = (x <= 1)
>>> c2 = (x >= 0)
>>> c3 = (sum(x) == 2)
\end{verbatim}

The built-in function \function{len()} returns the dimension of the
constraint function.

Constraints have four public attributes.
\begin{methoddesc}{type}
Returns \code{'='} if the constraint is an equality constraint,
and \code{'<'} if the constraint is an inequality constraint.
\end{methoddesc}

\begin{methoddesc}{value}
Returns the value of the constraint function.  
\end{methoddesc}

\begin{memberdesc}{multiplier}
For a constraint \var{c}, \var{c}.\member{multiplier} is a 
\pytype{variable} object of dimension \code{len(\var{c})}.   
It is used to represent the Lagrange multiplier or dual variable 
associated with the constraint.
Its value is initialized as \None, and can be modified
by making an assignment to \var{c}.\member{multiplier}.\member{value}.
\end{memberdesc}

\begin{memberdesc}{name}
The name of the constraint.  Changing the name of a constraint
also changes the name of the multiplier of \var{c}.
For example, the command \code{\var{c}.\member{name} = 'newname'} also 
changes
\var{c}.\member{multiplier}.\member{name} to \code{'newname\_mul'}.
\end{memberdesc}


\section{Optimization Problems} \label{s-lp}

Optimization problems are be constructed by calling the following
function.
\begin{classdesc}{op}{\optional{objective\optional{, constraints\optional{, name}}}}
The first argument specifies the objective function to be minimized.
It can be an affine or convex piecewise-linear function with length 1, 
a \pytype{variable} with length 1, or a scalar constant
(\intgr, \flt\ or 1 by 1 dense \dtc\ matrix).  The default value is
\code{0.0}.

The second argument is a single \pytype{constraint}, or a list of 
\pytype{constraint} objects.  The default value is an empty list.

The third argument is a string with a name for the problem.
The default value is the empty string.
\end{classdesc}

The following attributes and methods are useful for examining
and modifying optimization problems.

\begin{memberdesc}{objective}
The objective or cost function.  One can write to this 
attribute to change the objective of an existing problem.  
\end{memberdesc}

\begin{methoddesc}{variables}
Returns a list of the variables of the problem.
\end{methoddesc}

\begin{methoddesc}{constraints}
Returns a list of the constraints.
\end{methoddesc}

\begin{methoddesc}{inequalities}
Returns a list of the inequality constraints.
\end{methoddesc}

\begin{methoddesc}{equalities}
Returns a list of the equality constraints.
\end{methoddesc}

\begin{funcdesc}{delconstraint}{c}
Deletes constraint {\tt c} from the problem.
\end{funcdesc}

\begin{funcdesc}{addconstraint}{c}
Adds constraint {\tt c} to the problem.
\end{funcdesc}

An optimization problem with convex piecewise-linear objective and
constraints can be solved by calling the method \function{solve()}.

\begin{funcdesc}{solve}{\optional{format\optional{, solver}}} 
This function converts the optimization problem to a linear program
in matrix form and then solves it using the solver described in 
section~\ref{s-lpsolver}.

The first argument is either \code{'dense'} or \code{'sparse'}, and 
denotes the matrix types used in the matrix representation of the LP.
The default value is \code{'dense'}.

The second argument is either \None. \code{'glpk'} or \code{'mosek'},
and selects one of three available LP solvers: a default solver 
written in Python, the GLPK solver (if installed) or the
MOSEK LP solver (if installed); see section~\ref{s-lpsolver}.
The default value  is \None.

The solver reports the outcome of optimization by setting 
the attribute \var{self}.\member{status} and by modifying  
the \member{value} attributes of the variables and the constraint 
multipliers of the problem.
\BIT
\item If the problem is solved to optimality, 
\var{self}.\member{status} is set to \code{'optimal'}.  
The \member{value} attributes of the variables in the problem  are set 
to their computed solutions, and the \member{value} attributes of the 
multipliers of the constraints of the problem are set to the computed 
dual optimal solution.

\item If it is determined that the problem is infeasible, 
\var{self}.\member{status} is set to \code{'primal infeasible'}.  
The \member{value} attributes of the variables are set to \None.
The \member{value} attributes of the 
multipliers of the constraints of the problem are set to a certificate 
of primal infeasibility.
With the \code{'glpk'} option, \function{solve()}
does not provide certificates of infeasibility.

\item If it is determined that the problem is dual infeasible, 
\var{self}.\member{status} is set to \code{'dual infeasible'}.  
The \member{value} attributes of the multipliers of the constraints of 
the problem are set to \None.
The \member{value} attributes of the 
variables are set to a certificate of dual infeasibility. 
With the \code{'glpk'} option, \function{solve()} does not provide 
certificates of infeasibility.

\item If the problem was not solved successfully,
\var{self}.\member{status} is set to \code{'unknown'}.  
The \member{value} attributes of the variables and the constraint
multipliers are set to \None.
\EIT
\end{funcdesc}
We refer to section~\ref{s-lpsolver} for details on the algorithms and
the different solver options.

As an example we solve the LP
\[
 \begin{array}{ll}
  \mbox{minimize} & -4x - 5y \\
  \mbox{subject to} &  2x +y \leq 3 \\
 &  x +2y \leq 3 \\
 & x \geq 0, \quad y \geq 0.
 \end{array}
\]
\begin{verbatim}
>>> x = variable()
>>> y = variable()
>>> c1 = ( 2*x+y <= 3 ) 
>>> c2 = ( x+2*y <= 3 )
>>> c3 = ( x >= 0 )
>>> c4 = ( y >= 0 ) 
>>> lp1 = op(-4*x-5*y, [c1,c2,c3,c4]) 
>>> lp1.solve()
>>> lp1.status
'optimal'
>>> print lp1.objective.value()
[-9.00e+00]
>>> print x.value
[ 1.00e+00]
>>> print y.value
[ 1.00e+00]
>>> print c1.multiplier.value
[ 1.00e+00]
>>> print c2.multiplier.value
[ 2.00e+00]
>>> print c3.multiplier.value
[ 2.87e-08]
>>> print c4.multiplier.value
[ 2.80e-08]
\end{verbatim}

We can solve the same LP in  matrix form as follows.
\begin{verbatim}
>>> x = variable(2)
>>> A = matrix([[2.,1.,-1.,0.], [1.,2.,0.,-1.]])
>>> b = matrix([3.,3.,0.,0.])
>>> c = matrix([-4.,-5.])
>>> ineq = ( A*x <= b )
>>> lp2 = op(dot(c,x), ineq)
>>> lp2.solve()
>>> print lp2.objective.value()
[-9.00e+00]
>>> print x.value
[ 1.00e+00]
[ 1.00e+00]
>>> print ineq.multiplier.value
[1.00e+00]
[2.00e+00]
[2.87e-08]
[2.80e-08]
\end{verbatim}

The \pytype{op} class also includes two methods for writing and reading
files in 
\ulink{MPS format}{http://www-fp.mcs.anl.gov/otc/Guide/OptWeb/continuous/constrained/linearprog/mps.html}.

\begin{funcdesc}{tofile}{filename}
If the problem is an LP, writes it to the file \code{'filename'} using 
the MPS format.  Row and column labels are assigned based on the 
variable and constraint names in the LP.  
\end{funcdesc}

\begin{funcdesc}{fromfile}{filename}
Reads the LP from the file \code{'filename'}.  The file must be 
a fixed-format MPS file.  Some features of the MPS format are not 
supported: comments beginning with dollar signs,
the row types 'DE', 'DL', 'DG', and 'DN', and the capability of
reading multiple righthand side, bound or range vectors.
\end{funcdesc}


\section{Examples}

\begin{description}
\item[Norm and Penalty Approximation]

In the first example we solve the norm approximation problems
\[
 \begin{array}{ll} 
 \mbox{minimize} & \|Ax - b\|_\infty,
 \end{array} \qquad
 \begin{array}{ll} 
 \mbox{minimize} & \|Ax - b\|_1
 \end{array},
\]
and the penalty approximation problem
\[
 \begin{array}{ll} 
 \mbox{minimize} & \sum_k \phi((Ax-b)_k), 
 \end{array} \qquad
 \phi(u) = \left\{\begin{array}{ll}
    0 & |u| \leq 3/4 \\
   |u|-3/4  & 3/4 \leq |u| \leq 3/2 \\
   2|u|-9/4  & |u| \geq 3/2.\end{array}\right.
\]
We use randomly generated data.

The code uses the \ulink{Matplotlib}{http://matplotlib.sourceforge.net}
package for plotting the histograms of the residual vectors for the
two solutions.  It generates the figure shown below.

\begin{verbatim}
from cvxopt.base import normal
from cvxopt.modeling import variable, op, max, sum
import pylab

m, n = 500, 100
A = normal(m,n)
b = normal(m)

x1 = variable(n)
op(max(abs(A*x1-b))).solve()

x2 = variable(n)
op(sum(abs(A*x2-b))).solve()

x3 = variable(n)
op(sum(max(0, abs(A*x3-b)-0.75, 2*abs(A*x3-b)-2.25))).solve()

pylab.subplot(311)
pylab.hist(A*x1.value-b, m/5)
pylab.subplot(312)
pylab.hist(A*x2.value-b, m/5)
pylab.subplot(313)
pylab.hist(A*x3.value-b, m/5)
pylab.show()
\end{verbatim}

\begin{center}
\includegraphics[width=15cm]{figures/normappr.eps}
\end{center}

Equivalently, we can formulate and solve the problems as LPs.
\begin{verbatim}
t = variable()
x1 = variable(n)
op(t, [-t <= A*x1-b, A*x1-b<=t]).solve()

u = variable(m)
x2 = variable(n)
op(sum(u), [-u <= A*x2+b, A*x2+b <= u]).solve()

v = variable(m)
x3 = variable(n)
op(sum(v), [v >= 0, v >= A*x3+b-0.75, v >= -(A*x3+b)-0.75, v >= 2*(A*x3-b)-2.25, v >= -2*(A*x3-b)-2.25]).solve()
\end{verbatim}


\item[Robust Linear Programming]
The robust LP
\[
 \begin{array}{ll}
 \mbox{minimize} & c^T x \\
 \mbox{subject to} & \sup_{\|v\|_\infty \leq 1} 
    (a_i+v)^T x \leq b_i, \qquad i=1,\ldots,m
 \end{array}
\]
is equivalent to the problem
\[
 \begin{array}{ll}
 \mbox{minimize} & c^Tx \\
 \mbox{subject to} & a_i^Tx + \|x\|_1 \leq b_i, \qquad i=1,\ldots,m.
 \end{array}
\]
The following code computes the solution and the solution of
the equivalent LP
\[
 \begin{array}{ll}
 \mbox{minimize} & c^Tx \\
 \mbox{subject to} & a_i^Tx + \ones^Ty \leq b_i, \qquad i=1,\ldots,m \\
& -y \preceq x \preceq y
\end{array}
\]
for randomly generated data.

\begin{verbatim}
from cvxopt.base import normal, uniform
from cvxopt.modeling import variable, dot, op, sum 

m, n = 500, 100
A = normal(m,n)
b = uniform(m)
c = normal(n)

x = variable(n)
op(dot(c,x), A*x+sum(abs(x)) <= b).solve()

x2 = variable(n)
y = variable(n)
op(dot(c,x2), [A*x2+sum(y) <= b, -y <= x2, x2 <= y]).solve()
\end{verbatim}


\item[1-Norm Support Vector Classifier]

The following problem arises in classification:
\[
\begin{array}{ll}
\mbox{minimize} & \|x\|_1 + \ones^Tu \\
\mbox{subject to} & Ax \succeq \ones -u \\
& u \succeq 0.
\end{array}
\]
It can be solved as follows.
\begin{verbatim}
x = variable(A.size[1],'x')
u = variable(A.size[0],'u')
op(sum(abs(x)) + sum(u), [A*x >= 1-u, u >= 0]).solve()
\end{verbatim}
An equivalent unconstrained formulation is
\begin{verbatim}
x = variable(A.size[1],'x')
op(sum(abs(x)) + sum(max(0,1-A*x))).solve()
\end{verbatim}
\end{description}
