A multi-convex optimization problem is one in which the variables can be partitioned into sets, over each of which the problem is convex when the other variables are fixed. It is generally a nonconvex problem. DMCP package provides methods to verify multi-convexity and to find minimal sets of variables that have to be fixed for the problem to be convex, as well as an organized heuristic for multi-convex programming. The full details of our approach are discussed in the associated paper. DMCP is built on top of CVXPY, a domain-specific language for convex optimization embedded in Python.
You should first install CVXPY 1.0. Then clone the repository and run python setup.py install
inside.
Consider an optimization problem
minimize f_0(x)
subject to f_i(x) <= 0, i = 1,...,m
g_i(x) = 0, i = 1,...,p,
where variable x
admits a partition of blocks of variables x = (x_1,...,x_N)
, and functions f_i
for i = 0,...,m
and g_i
for i = 1,...,p
are proper.
Given a set of DCP atomic functions and its extension of multi-convex atomic functions,
the problem can be specified as disciplined multi-convex programming (DMCP), if there are index sets F_1,..., F_K
, such that their intersection is empty, and for every k
the problem with variables x_i
for all i
in set F_k
fixed to any value can be specified as DCP with respect to the DCP atoms set.
The following code uses DMCP to approximately solve a simple multi-convex problem.
import cvxpy as cvx
import dmcp
x_1 = cvx.Variable()
x_2 = cvx.Variable()
x_3 = cvx.Variable()
x_4 = cvx.Variable()
objective = cvx.Minimize(cvx.abs(x_1*x_2+x_3*x_4))
constraint = [x_1+x_2+x_3+x_4 == 1]
myprob = cvx.Problem(objective, constraint)
print("minimal sets:", dmcp.find_minimal_sets(myprob)) # find all minimal sets
print("problem is DCP:", myprob.is_dcp()) # false
print("problem is DMCP:", dmcp.is_dmcp(myprob)) # true
result = myprob.solve(method = 'bcd')
The output of the above code is as follows.
minimal sets: [[1, 3], [1, 2], [0, 3], [0, 2]]
problem is DCP: False
problem is DMCP: True
maximum value of slack variables: 1.15081491391e-05
objective value: 1.74866042578e-05
The solutions obtained by DMCP can depend on the initial point.
The algorithm starts from the values of any variables that are already specified;
for any that are not specified, random values are used.
It is suggested that users set reasonable initial values for all variables,
which can be done by manually setting the value
field of the problem variables.
For example:
x_1.value = 1.2
x_2.value = -3
x_3.value = 4
x_4.value = 0.15
result = myprob.solve(method = 'bcd')
More examples can be found [here] (https://github.com/cvxgrp/dmcp/tree/master/examples).
In order to allow multi-convex functions, we extend the atomic function set of CVXPY
.
The following atoms are allowed to have non-constant expressions in both arguments, while in the dictionary of CVXPY
the first argument must be constant.
- multiplication:
expression1 * expression2
- elementwise multiplication:
cvx.multiply(expression1, expression2)
is_dmcp(problem)
returns a boolean indicating if an optimization problem satisfies DMCP rules.find_minimal_sets(problem)
analyzes the problem and returns a list of minimal sets of (indexes of) variables. The indexes are with respect to the listproblem.variables()
, namely the variable corresponding to the index0
isproblem.variables()[0]
. If the problem is DCP, it returns an empty list.is_all = True
will generate all minimal sets.fix(obj, fix_vars)
returns a new expression or a new problem with the variables in the listfix_vars
replaced with parameters of the same dimensions and signs. Theobj
can either be an expression or a problem.
The components of the variable, the objective, and the constraints are constructed using standard CVXPY syntax. Once the user has constructed a problem object, they can apply the following solve method:
problem.solve(method = 'bcd')
applies the solving algorithm with proximal operators, and returns the number of iterations, and the maximum value of the slack variables. The solution to every variable is in itsvalue
field.problem.solve(method = 'bcd', update = 'minimize')
applies the solving method without proximal operators.problem.solve(method = 'bcd', update = 'prox_linear')
applies the solving method with prox-linear operators.
Additional arguments can be used to specify the parameters.
Solve method parameters:
- The
solver
parameter specifies what solver to use to solve convex subproblems. - The
max_iter
parameter sets the maximum number of iterations in the algorithm. The default is 100. - The
mu
parameter trades off satisfying the constraints and minimizing the objective. Largermu
favors satisfying the constraints. The default is 0.001. - The
rho
parameter sets the rate at whichmu
increases inside the algorithm. The default is 1.2. - The
mu_max
parameter upper bounds how largemu
can get. The default is 1e4. - The
lambd
parameter is the parameter in the proximal operator. The default is 10.
If the convex solver for the subproblems accepts any additional keyword arguments,
then the user can set them in the problem.solve()
function, and they will be passed to the convex solver.