ProductOfSets is wrong when a constraint has zero rows

[odow]

This took me a while to track down. It is the cause of the failure in jump-dev/JuMP.jl#4188 (comment)

The chain goes:

• PiecewiseLinearOpt attempts to solve a convex MINLP
• It uses Pajarito, with HiGHS + COSMO
• Pajarito doesn't have special handling for zero-dim rows. It tries to modify them as if they're normal: https://github.com/jump-dev/Pajarito.jl/blob/83048f2225eac4b8d4226a9c686658b885b2b6d7/src/models.jl#L209-L214
• COSMO doesn't support modification, so we throw away the optimizer and try to modify the cache
• The cache says it doesn't have this constraint, even though we previously added it via add_constraint

Phew!

Here's a MWE:

julia> import MathOptInterface as MOI

julia> MOI.Utilities.@product_of_sets(_Cones, MOI.Zeros)

julia> const OptimizerCache{T} = MOI.Utilities.GenericModel{
           T,
           MOI.Utilities.ObjectiveContainer{T},
           MOI.Utilities.VariablesContainer{T},
           MOI.Utilities.MatrixOfConstraints{
               T,
               MOI.Utilities.MutableSparseMatrixCSC{
                   T,
                   Int,
                   MOI.Utilities.OneBasedIndexing,
               },
               Vector{T},
               _Cones{T},
           },
       };

julia> model = OptimizerCache{Float64}();

julia> f = MOI.VectorAffineFunction(MOI.VectorAffineTerm{Float64}[], Float64[]);

julia> c = MOI.add_constraint(model, f, MOI.Zeros(0))
MathOptInterface.ConstraintIndex{MathOptInterface.VectorAffineFunction{Float64}, MathOptInterface.Zeros}(1)

julia> MOI.Utilities.final_touch(model, nothing)

julia> MOI.get(model, MOI.ListOfConstraintTypesPresent())
Tuple{Type, Type}[]

julia> F, S = MOI.VectorAffineFunction{Float64}, MOI.Zeros
(MathOptInterface.VectorAffineFunction{Float64}, MathOptInterface.Zeros)

julia> MOI.get(model, MOI.NumberOfConstraints{F,S}())
0

julia> MOI.get(model, MOI.ListOfConstraintIndices{F,S}())
MathOptInterface.ConstraintIndex{MathOptInterface.VectorAffineFunction{Float64}, MathOptInterface.Zeros}[]

[odow]

So this is actually a problem with ProductOfSets:

julia> import MathOptInterface as MOI

julia> using Test

julia> MOI.Utilities.@product_of_sets(VectorSets, MOI.Nonnegatives)

julia> F, S = MOI.VectorAffineFunction{Int}, MOI.Nonnegatives
(MathOptInterface.VectorAffineFunction{Int64}, MathOptInterface.Nonnegatives)

julia> @testset "A" begin
           sets = VectorSets{Int}()
           i = MOI.Utilities.set_index(sets, S)
           MOI.Utilities.add_set(sets, i, 0)
           MOI.Utilities.final_touch(sets)
           @test (F, S) in MOI.get(sets, MOI.ListOfConstraintTypesPresent())
           @test MOI.get(sets, MOI.NumberOfConstraints{F,S}()) == 1
           c = MOI.ConstraintIndex{F,S}.(1:1)
           @test MOI.get(sets, MOI.ListOfConstraintIndices{F,S}()) == c
           @test all(MOI.is_valid(sets, ci) for ci in c)
       end
A: Test Failed at REPL[76]:6
  Expression: (F, S) in MOI.get(sets, MOI.ListOfConstraintTypesPresent())
   Evaluated: (MathOptInterface.VectorAffineFunction{Int64}, MathOptInterface.Nonnegatives) in Tuple{Type, Type}[]

Stacktrace:

A: Test Failed at REPL[76]:7
  Expression: MOI.get(sets, MOI.NumberOfConstraints{F, S}()) == 1
   Evaluated: 0 == 1

Stacktrace:

A: Test Failed at REPL[76]:9
  Expression: MOI.get(sets, MOI.ListOfConstraintIndices{F, S}()) == c
   Evaluated: MathOptInterface.ConstraintIndex{MathOptInterface.VectorAffineFunction{Int64}, MathOptInterface.Nonnegatives}[] == MathOptInterface.ConstraintIndex{MathOptInterface.VectorAffineFunction{Int64}, MathOptInterface.Nonnegatives}[MathOptInterface.ConstraintIndex{MathOptInterface.VectorAffineFunction{Int64}, MathOptInterface.Nonnegatives}(1)]

Stacktrace:

A: Test Failed at REPL[76]:10
  Expression: all((MOI.is_valid(sets, ci) for ci = c))

Stacktrace:

Test Summary: | Fail  Total  Time
A             |    4      4  0.0s
RNG of the outermost testset: Random.Xoshiro(0x9fa7d8cebe81f901, 0xc2272e9fdfe46f03, 0xe8bffbd11d93aff9, 0x498d4a92bd73bbf7, 0x74ba59c7203bd9a8)
ERROR: Some tests did not pass: 0 passed, 4 failed, 0 errored, 0 broken.

julia> @testset "B" begin
           sets = VectorSets{Int}()
           i = MOI.Utilities.set_index(sets, S)
           MOI.Utilities.add_set(sets, i, 0)
           MOI.Utilities.add_set(sets, i, 2)
           MOI.Utilities.add_set(sets, i, 0)
           MOI.Utilities.add_set(sets, i, 1)
           MOI.Utilities.add_set(sets, i, 0)
           MOI.Utilities.final_touch(sets)
           @test (F, S) in MOI.get(sets, MOI.ListOfConstraintTypesPresent())
           @test MOI.get(sets, MOI.NumberOfConstraints{F,S}()) == 5
           c = MOI.ConstraintIndex{F,S}.(1:5)
           @test MOI.get(sets, MOI.ListOfConstraintIndices{F,S}()) == c
           @test all(MOI.is_valid(sets, ci) for ci in c)
       end
B: Test Failed at REPL[77]:11
  Expression: MOI.get(sets, MOI.NumberOfConstraints{F, S}()) == 5
   Evaluated: 2 == 5

Stacktrace:

B: Test Failed at REPL[77]:13
  Expression: MOI.get(sets, MOI.ListOfConstraintIndices{F, S}()) == c
   Evaluated: MathOptInterface.ConstraintIndex{MathOptInterface.VectorAffineFunction{Int64}, MathOptInterface.Nonnegatives}[MathOptInterface.ConstraintIndex{MathOptInterface.VectorAffineFunction{Int64}, MathOptInterface.Nonnegatives}(1), MathOptInterface.ConstraintIndex{MathOptInterface.VectorAffineFunction{Int64}, MathOptInterface.Nonnegatives}(3)] == MathOptInterface.ConstraintIndex{MathOptInterface.VectorAffineFunction{Int64}, MathOptInterface.Nonnegatives}[MathOptInterface.ConstraintIndex{MathOptInterface.VectorAffineFunction{Int64}, MathOptInterface.Nonnegatives}(1), MathOptInterface.ConstraintIndex{MathOptInterface.VectorAffineFunction{Int64}, MathOptInterface.Nonnegatives}(2), MathOptInterface.ConstraintIndex{MathOptInterface.VectorAffineFunction{Int64}, MathOptInterface.Nonnegatives}(3), MathOptInterface.ConstraintIndex{MathOptInterface.VectorAffineFunction{Int64}, MathOptInterface.Nonnegatives}(4), MathOptInterface.ConstraintIndex{MathOptInterface.VectorAffineFunction{Int64}, MathOptInterface.Nonnegatives}(5)]

Stacktrace:

B: Test Failed at REPL[77]:14
  Expression: all((MOI.is_valid(sets, ci) for ci = c))

Stacktrace:

Test Summary: | Pass  Fail  Total  Time
B             |    1     3      4  0.0s
RNG of the outermost testset: Random.Xoshiro(0x9fa7d8cebe81f901, 0xc2272e9fdfe46f03, 0xe8bffbd11d93aff9, 0x498d4a92bd73bbf7, 0x74ba59c7203bd9a8)
ERROR: Some tests did not pass: 1 passed, 3 failed, 0 errored, 0 broken.

[odow]

So fixing this requires quite a large change to ProductOfSets. I need to think about the best approach.

In the near-term, I'm fixing Pajarito to stop it modifying zero-dimensional constraints: jump-dev/Pajarito.jl#460