chore: Updating VectorStore batch size to improve performance

[jamie-ons]

✨ Summary

VectorStore previously exposed batch_size as a repeated parameter on individual methods, creating multiple independent sources of truth. This PR consolidates that to a single value set at construction time.

To inform the choice of default, a profiling analysis was run across the target GCP instance range at batch sizes from 8 to 512. The default has been updated to the value that minimises search time without risking OOM on the smallest supported instances.

Constraints: must not break or perform significantly worse on 2 vCPU instances; optimised for typical cloud deployments at 4–8 vCPUs.

📜 Changes Introduced

• VectorStore methods updated so batch_size is self.batch_size from the constructor.
• Profiling analysis across e2-standard-2, e2-medium, and e2-standard-8 measuring latency and memory
• Default batch_size updated from 8 to based on analysis findings

✅ Checklist

• Code passes linting with Ruff
• DocStrings follow Google-style and are added as per Pylint recommendations
• Documentation has been updated if needed

🔍 How to Test

To test this code, run the DEMO/general_workflow_demo.ipynb.

To test that it does not break on small instances, run on the following GCP instances and locally:

• e2-standard-2
• e2-medium
• e2-standard-8

[lukeroantreeONS]

I think we'd like to retain the option for users to specify a different batch size at this point, but we'd want the default behaviour to follow the single source of truth.