feat: groundedness requirement

[akihikokuroda]

Misc PR

Type of PR

• Bug Fix
• New Feature
• Documentation
• Other

Description

• Link to Issue: Fixes IVR Groundedness Validator Using Citations #775

Testing

• Tests added to the respective file if code was changed
• New code has 100% coverage if code as added
• Ensure existing tests and github automation passes (a maintainer will kick off the github automation when the rest of the PR is populated)

[github-actions]

The PR description has been updated. Please fill out the template for your PR to be reviewed.

[jakelorocco]

This is a very interesting requirement. I think it's a good opportunity to show off Mellea intrinsics and requirement checking. I'm not sure we have many other requirements with as many llm calls.

One broader suggestion than the comments I left below: Could we parallelize the steps? Could we generate citations at the same time we check spans for needing citations? As we generate spans that need to be checked, could we check each in parallel or as they are given? If so, I think we should make this requirement work more asynchronously and have an early exit mode if a span fails the check (even if not all citations have been generated / not all spans have been checked).

[psschwei]

cc @generative-computing/mellea-intrinsics

[akihikokuroda]

@jakelorocco Thanks for review. I addressed all your comments except "Could we parallelize the steps?". I'm working on it.

[akihikokuroda]

@jakelorocco There are 2 ideas improve the requirement.
For this:
OPTIMIZED_PIPELINE_DESIGN.md
I'm checking if citation intrinsic works in this usage.

This one does not parallelize the processing but it make a batch call for the citation support step.
COMBINED_SUPPORT_ASSESSMENT_DESIGN.md

[akihikokuroda]

The "parallelize" seems some work/investigation necessary. So I improved "citation support" step to make only one LLM call instead of calling LLM for each span.