8. Learning Solver¶
On previous pages, we discussed various components of the MIPLearn framework, including training data collectors, feature extractors, and individual machine learning components. In this page, we introduce LearningSolver, the main class of the framework which integrates all the aforementioned components into a cohesive whole. Using LearningSolver involves three steps: (i) configuring the solver; (ii) training the ML components; and (iii) solving new MIP instances. In the following, we describe each of these steps, then conclude with a complete runnable example.
8.1. Configuring the solver¶
LearningSolver is composed by multiple individual machine learning components, each targeting a different part of the solution process, or implementing a different machine learning strategy. This architecture allows strategies to be easily enabled, disabled or customized, making the framework flexible. By default, no components are provided and LearningSolver is equivalent to a traditional MIP solver. To specify additional components, the components constructor argument may be used:
solver = LearningSolver(
components=[
comp1,
comp2,
comp3,
]
)
In this example, three components comp1, comp2 and comp3 are provided. The strategies implemented by these components are applied sequentially when solving the problem. For example, comp1 and comp2 could fix a subset of decision variables, while comp3 constructs a warm start for the remaining problem.
8.2. Training and solving new instances¶
Once a solver is configured, its ML components need to be trained. This can be achieved by the solver.fit method, as illustrated below. The method accepts a list of HDF5 files and trains each individual component sequentially. Once the solver is trained, new instances can be solved using solver.optimize. The method returns a dictionary of statistics collected by each component, such as the number of variables fixed.
# Build instances
train_data = ...
test_data = ...
# Collect training data
bc = BasicCollector()
bc.collect(train_data, build_model)
# Build solver
solver = LearningSolver(...)
# Train components
solver.fit(train_data)
# Solve a new test instance
stats = solver.optimize(test_data[0], build_model)
8.3. Complete example¶
In the example below, we illustrate the usage of LearningSolver by building instances of the Traveling Salesman Problem, collecting training data, training the ML components, then solving a new instance.
[3]:
import random
import numpy as np
from scipy.stats import uniform, randint
from sklearn.linear_model import LogisticRegression
from miplearn.classifiers.minprob import MinProbabilityClassifier
from miplearn.classifiers.singleclass import SingleClassFix
from miplearn.collectors.basic import BasicCollector
from miplearn.components.primal.actions import SetWarmStart
from miplearn.components.primal.indep import IndependentVarsPrimalComponent
from miplearn.extractors.AlvLouWeh2017 import AlvLouWeh2017Extractor
from miplearn.io import write_pkl_gz
from miplearn.problems.tsp import (
TravelingSalesmanGenerator,
build_tsp_model,
)
from miplearn.solvers.learning import LearningSolver
# Set random seed to make example reproducible.
random.seed(42)
np.random.seed(42)
# Generate a few instances of the traveling salesman problem.
data = TravelingSalesmanGenerator(
n=randint(low=10, high=11),
x=uniform(loc=0.0, scale=1000.0),
y=uniform(loc=0.0, scale=1000.0),
gamma=uniform(loc=0.90, scale=0.20),
fix_cities=True,
round=True,
).generate(50)
# Save instance data to data/tsp/00000.pkl.gz, data/tsp/00001.pkl.gz, ...
all_data = write_pkl_gz(data, "data/tsp")
# Split train/test data
train_data = all_data[:40]
test_data = all_data[40:]
# Collect training data
bc = BasicCollector()
bc.collect(train_data, build_tsp_model, n_jobs=4)
# Build learning solver
solver = LearningSolver(
components=[
IndependentVarsPrimalComponent(
base_clf=SingleClassFix(
MinProbabilityClassifier(
base_clf=LogisticRegression(),
thresholds=[0.95, 0.95],
),
),
extractor=AlvLouWeh2017Extractor(),
action=SetWarmStart(),
)
]
)
# Train ML models
solver.fit(train_data)
# Solve a test instance
solver.optimize(test_data[0], build_tsp_model)
Gurobi Optimizer version 10.0.1 build v10.0.1rc0 (linux64)
CPU model: AMD Ryzen 9 7950X 16-Core Processor, instruction set [SSE2|AVX|AVX2|AVX512]
Thread count: 16 physical cores, 32 logical processors, using up to 32 threads
Optimize a model with 10 rows, 45 columns and 90 nonzeros
Model fingerprint: 0x6ddcd141
Coefficient statistics:
Matrix range [1e+00, 1e+00]
Objective range [4e+01, 1e+03]
Bounds range [1e+00, 1e+00]
RHS range [2e+00, 2e+00]
Presolve time: 0.00s
Presolved: 10 rows, 45 columns, 90 nonzeros
Iteration Objective Primal Inf. Dual Inf. Time
0 6.3600000e+02 1.700000e+01 0.000000e+00 0s
15 2.7610000e+03 0.000000e+00 0.000000e+00 0s
Solved in 15 iterations and 0.00 seconds (0.00 work units)
Optimal objective 2.761000000e+03
Set parameter LazyConstraints to value 1
Gurobi Optimizer version 10.0.1 build v10.0.1rc0 (linux64)
CPU model: AMD Ryzen 9 7950X 16-Core Processor, instruction set [SSE2|AVX|AVX2|AVX512]
Thread count: 16 physical cores, 32 logical processors, using up to 32 threads
Optimize a model with 10 rows, 45 columns and 90 nonzeros
Model fingerprint: 0x74ca3d0a
Variable types: 0 continuous, 45 integer (45 binary)
Coefficient statistics:
Matrix range [1e+00, 1e+00]
Objective range [4e+01, 1e+03]
Bounds range [1e+00, 1e+00]
RHS range [2e+00, 2e+00]
User MIP start produced solution with objective 2796 (0.00s)
Loaded user MIP start with objective 2796
Presolve time: 0.00s
Presolved: 10 rows, 45 columns, 90 nonzeros
Variable types: 0 continuous, 45 integer (45 binary)
Root relaxation: objective 2.761000e+03, 14 iterations, 0.00 seconds (0.00 work units)
Nodes | Current Node | Objective Bounds | Work
Expl Unexpl | Obj Depth IntInf | Incumbent BestBd Gap | It/Node Time
0 0 2761.00000 0 - 2796.00000 2761.00000 1.25% - 0s
0 0 cutoff 0 2796.00000 2796.00000 0.00% - 0s
Cutting planes:
Lazy constraints: 3
Explored 1 nodes (16 simplex iterations) in 0.01 seconds (0.00 work units)
Thread count was 32 (of 32 available processors)
Solution count 1: 2796
Optimal solution found (tolerance 1.00e-04)
Best objective 2.796000000000e+03, best bound 2.796000000000e+03, gap 0.0000%
User-callback calls 110, time in user-callback 0.00 sec
[3]:
{'WS: Count': 1, 'WS: Number of variables set': 41.0}
[ ]: