Use ec2.Route instead of inline routes for VPC component

[weston-harper-rox]

There is a known limitation of the AWS provider that prevents inline routes and ec2.Route constructs from working reliably inside a RouteTable definition.

This prevents consumers of the VPC component from adding routes to the VPC's route tables. We'd like to do this to establish a peering connection between accounts for cross-account databases, where the peering connection allows for ingestion of our application database tables into our data warehouse.

Here's a dummy example:

const vpc = new sst.aws.Vpc('VPC', {
  bastion: true,
  nat: 'ec2',
});

vpc.nodes.privateRouteTables.apply((routeTables) => {
  routeTables.map((table, idx) => {
    new aws.ec2.Route(`PeeringRoute${idx}`, {
      routeTableId: table.id,
      vpcPeeringConnectionId: 'pcx-foo',
      destinationCidrBlock: '9.9.9.9/16',
    });
  });
});

While this works some of the time, the limitation leads to state sync errors that bring the peering connection offline and force us to manually rectify.

Request

Use ec2.Route instead of inline routes in createPublicSubnets and createPrivateSubnets in the vpc component.

[vimtor]

@weston-harper-rox we can close this issue now?

what was the specific error that you were getting?

[weston-harper-rox]

@vimtor yes, thanks for taking a look. Basically, I wanted to use the pattern above, but since the VPC component uses inline routes, ec2.Route will result in known issues. When I was working on a solution, I realized that updating the SST component is impossible without end-user headaches because you need to first clear the routes using [] as a value, then reinstate them as ec2.Routes.

The workaround for our team was to use pulumi transforms like so:

new sst.aws.Vpc(
  'VPC',
  {
    bastion: true,
    nat: 'ec2',
  },
  {
    transforms: [
      (args) => {
        if (args.type === 'aws:ec2/routeTable:RouteTable' && args.name.includes('Private')) {
          return {
            ...args,
            props: {
              ...args.props,
              routes: [
                ...args.props.routes,
                {
                  vpcPeeringConnectionId,
                  cidrBlock,
                },
              ],
            },
          };
        }
      },
    ],
  },
)

We used the same approach to resolve our issues in #6604, so I'll close that as well.

[vimtor]

@weston-harper-rox will everyone using sst face those issues or are they related to your specific setup?

[weston-harper-rox]

@vimtor anyone using SST who wants to add routes to any of the component's route tables or ingress/egress rules to the component's default security group will run into these issues.

They would not have the issue if they:

• Choose to override the route tables or security group entirely using SST's transform object
  • This means they have to re-implement the default settings if desired and that they will no longer get updates to the defaults if changes are made to the construct
• Choose to make and attach new route tables/security groups in addition to the default ones, which can be quite a pain to set up

However, given the headaches required in migrating, I'd imagine the best path forward would be to make a v3 vpc component along with a migration guide. Happy to help with that if you'd like.