While trying to get a different game to run under Butterscotch, I found that an assertion in vm.c require(VM_STACK_SIZE > ctx->stack.top); was getting triggered. After tracing the offending GML code, I found that creating a new instance within a repeat loop would loop indefinitely and cause that requirement to throw.
Below is a minimal example I created to showcase this behavior:
function WithinLoopInstance() constructor
{
}
function LoopInConstructor(arg0) constructor
{
size = arg0;
show_debug_message("Loop begin");
repeat (arg0)
{
show_debug_message("Loop itter");
var temp = new WithinLoopInstance();
}
}
show_debug_log(true);
var myLoopInstance = new LoopInConstructor(10);
Using the UndertaleModTool, this generates the following assembly
:[0]
b [2]
> gml_Script_WithinLoopInstance@gml_Room_Room1_Create (locals=0, argc=0)
:[1]
call.i @@SetStatic@@(argc=0)
exit.i
:[2]
push.i [function]gml_Script_WithinLoopInstance@gml_Room_Room1_Create
conv.i.v
call.i @@NullObject@@(argc=0)
call.i method(argc=2)
dup.v 0
pop.v.v self.WithinLoopInstance
popz.v
b [6]
> gml_Script_LoopInConstructor@gml_Room_Room1_Create (locals=1, argc=1)
:[3]
call.i @@SetStatic@@(argc=0)
push.v arg.argument0
pop.v.v builtin.size
push.s "Loop begin"@59
conv.s.v
call.i show_debug_message(argc=1)
popz.v
push.v arg.argument0
conv.v.i
dup.i 0
push.i 0
cmp.i.i LTE
bt [5]
:[4]
push.s "Loop itter"@60
conv.s.v
call.i show_debug_message(argc=1)
popz.v
pushref.i gml_Script_WithinLoopInstance@gml_Room_Room1_Create
call.i @@NewGMLObject@@(argc=1)
pop.v.v local.temp
pushi.e 1
sub.i.i
dup.i 0
bt [4]
:[5]
popz.i
exit.i
:[6]
push.i [function]gml_Script_LoopInConstructor@gml_Room_Room1_Create
conv.i.v
call.i @@NullObject@@(argc=0)
call.i method(argc=2)
dup.v 0
pop.v.v self.LoopInConstructor
popz.v
pushi.e 1
conv.b.v
call.i show_debug_log(argc=1)
popz.v
pushi.e 10
conv.i.v
push.i [function]gml_Script_LoopInConstructor@gml_Room_Room1_Create
conv.i.v
call.i @@NewGMLObject@@(argc=2)
pop.v.v local.myLoopInstance
:[end]
When I traced to [4] (inside the generated repeat loop) and checked the stack prior during the subtraction operation, I found that @@setstatic@@ within the other constructor function had pushed a value that the subtraction operation was reading. As a result, the repeat loop would run infinitely until the stack overflowed.
While trying to get a different game to run under Butterscotch, I found that an assertion in vm.c require(VM_STACK_SIZE > ctx->stack.top); was getting triggered. After tracing the offending GML code, I found that creating a new instance within a repeat loop would loop indefinitely and cause that requirement to throw.
Below is a minimal example I created to showcase this behavior:
Using the UndertaleModTool, this generates the following assembly
When I traced to [4] (inside the generated repeat loop) and checked the stack prior during the subtraction operation, I found that @@setstatic@@ within the other constructor function had pushed a value that the subtraction operation was reading. As a result, the repeat loop would run infinitely until the stack overflowed.