docs: Improve getting started and testing guides for humans and agents by alamb · Pull Request #20970 · apache/datafusion

alamb · 2026-03-16T19:30:41Z

Which issue does this PR close?

Closes #.

Rationale for this change

This PR is a follow-up to #20939 from @Dandandan.

The goal is to make it easier for both humans and agents to get started making changes in this repository and to create pull requests efficiently.

The repository already had the necessary contributor information, but it was spread across multiple documents and not easy to discover quickly.

What changes are included in this PR?

This PR makes the most important setup, testing, and pre-PR checks easier to find from the contributor guide and from AGENTS.md:

Add a quick-start setup section to the contributor guide with the shortest path to a working local environment.
Add a testing quick-start section summarizing the most important tests to run before submitting a PR.
Add a “Before Submitting a PR” section to centralize formatting and lint guidance.
Update AGENTS.md to point to the canonical contributor guide sections instead of duplicating setup and testing instructions.

Are these changes tested?

This PR updates documentation only.

Are there any user-facing changes?

This improves contributor-facing documentation and makes setup / testing guidance easier to discover, but it does not change DataFusion runtime behavior or public APIs.

alamb · 2026-03-16T19:31:02Z

-## Windows setup
+## Quick Start
+
+For the fastest path to a working local environment, follow these steps


I pulled most of this out of agents.md and left a link there instead

alamb · 2026-03-16T19:31:16Z

 Alternatively a binary release can be downloaded from the [Release Page](https://github.com/protocolbuffers/protobuf/releases) or [built from source](https://github.com/protocolbuffers/protobuf/blob/main/src/README.md).

-## Bootstrap environment
+## Bootstrap Environment


Made heading consistent.

alamb · 2026-03-16T19:31:30Z

 DataFusion has support for [dev containers](https://containers.dev/) which may be used for
 developing DataFusion in an isolated environment either locally or remote if desired. Using dev containers for developing
-DataFusion is not a requirement by any means but is available for those where doing local development could be tricky
+DataFusion is not a requirement but is available where doing local development could be tricky


drive by cleanup to make this more concise

alamb · 2026-03-16T19:32:07Z

 is not accidentally broken during refactorings. All new features
 should have test coverage and the entire test suite is run as part of CI.

+## Testing Quick Start


THis is based on what started in AGENTS.md but I made it slightly easier to understand

alamb · 2026-03-16T19:32:19Z

- `cargo clippy` catches common mistakes and enforces idiomatic Rust patterns. All warnings must be resolved (treated as errors via `-D warnings`).
-
-Do not commit code that fails either of these checks.
+See [Before Submitting a PR](docs/source/contributor-guide/index.md#before-submitting-a-pr)


moved and expanded content

My experience is agents will read the .md file on start, but will not always go read all the materials linked - still helps because they are found by grep etc. tools.

So to make sure they read things like cargo fmt you'll have to put it in the AGENTS.md

Perhaps keep the duplication for now?

Or perhaps we can try to verify this works (or be more explicit it must read docs/source/contributor-guide/index.md#before-submitting-a-pr before PRing.

Added in 350b910

Dandandan · 2026-03-17T04:46:52Z

-
-Do not commit code that fails either of these checks.
+See [Before Submitting a PR](docs/source/contributor-guide/index.md#before-submitting-a-pr)
+for the required formatting and lint checks.


Perhaps also link the PR template before creating a PR?

https://github.com/apache/datafusion/blob/main/.github/pull_request_template.md

Added in 350b910

adriangb

Overall looks great! It's always a bit hard to see how agents will react to these instructions. I see Daniel left some feedback, we can address before merging but can also merge and tweak based on reports of real world results / things that agents seem to get wrong repeatedly.

…_for_agents

2010YOUY01

Thank you, good to see an AGENTS.md! Also left some small suggestions.

2010YOUY01 · 2026-03-19T03:34:49Z

+./ci/scripts/rust_fmt.sh
+./ci/scripts/rust_clippy.sh


Suggested change

./ci/scripts/rust_fmt.sh

./ci/scripts/rust_clippy.sh

./dev/rust_lint.sh

# use the `--write` flag to automatically fix some formatting and lint errors

# ./dev/rust_lint.sh --write --allow-dirty

This script is the entry point for all non-functional tests. It includes the previous two scripts as well as several others.

Great call. Updated in 6411347

Co-authored-by: Yongting You <2010youy01@gmail.com>

…_for_agents

alamb · 2026-03-22T12:19:54Z

I feel like this is now an improvement over what is currently on main so I will merge it. Let's keep iterating and improving in follow on PRs. Thanks again for the reviews

@Dandandan

apache#20970) ## Which issue does this PR close? - Closes #. ## Rationale for this change This PR is a follow-up to apache#20939 from @Dandandan. The goal is to make it easier for both humans and agents to get started making changes in this repository and to create pull requests efficiently. The repository already had the necessary contributor information, but it was spread across multiple documents and not easy to discover quickly. ## What changes are included in this PR? This PR makes the most important setup, testing, and pre-PR checks easier to find from the contributor guide and from `AGENTS.md`: - Add a quick-start setup section to the contributor guide with the shortest path to a working local environment. - Add a testing quick-start section summarizing the most important tests to run before submitting a PR. - Add a “Before Submitting a PR” section to centralize formatting and lint guidance. - Update `AGENTS.md` to point to the canonical contributor guide sections instead of duplicating setup and testing instructions. ## Are these changes tested? This PR updates documentation only. ## Are there any user-facing changes? This improves contributor-facing documentation and makes setup / testing guidance easier to discover, but it does not change DataFusion runtime behavior or public APIs. --------- Co-authored-by: Yongting You <2010youy01@gmail.com>

Improve getting started / testing guides for Humans and Agents

e9fceb7

alamb added the documentation Improvements or additions to documentation label Mar 16, 2026

alamb commented Mar 16, 2026

View reviewed changes

touchups

2ed084f

alamb marked this pull request as ready for review March 16, 2026 19:34

alamb added 2 commits March 16, 2026 15:36

clean

20db17d

Merge branch 'main' into alamb/update_dev_docs_for_agents

83e2793

Dandandan reviewed Mar 17, 2026

View reviewed changes

Dandandan changed the title ~~docs: Improve getting started and testing guides for humans and hgents~~ docs: Improve getting started and testing guides for humans and agents Mar 17, 2026

alamb mentioned this pull request Mar 18, 2026

Add Local Scripts to Reproduce Full CI and Perform Auto-Fixes #19227

Closed

adriangb approved these changes Mar 18, 2026

View reviewed changes

alamb added 2 commits March 18, 2026 14:44

Stronger AGENTS.md instructions

350b910

Merge remote-tracking branch 'apache/main' into alamb/update_dev_docs…

014b72e

…_for_agents

2010YOUY01 reviewed Mar 19, 2026

View reviewed changes

Comment thread docs/source/contributor-guide/testing.md Outdated

2010YOUY01 reviewed Mar 19, 2026

View reviewed changes

alamb and others added 3 commits March 19, 2026 11:38

Update docs/source/contributor-guide/testing.md

37d8daa

Co-authored-by: Yongting You <2010youy01@gmail.com>

Merge remote-tracking branch 'apache/main' into alamb/update_dev_docs…

d55f664

…_for_agents

Update to use rust_lint rather

6411347

alamb added this pull request to the merge queue Mar 22, 2026

Merged via the queue into apache:main with commit 878b879 Mar 22, 2026
8 checks passed

alamb deleted the alamb/update_dev_docs_for_agents branch March 22, 2026 12:20

Dandandan mentioned this pull request Mar 26, 2026

docs: Add explicit fmt and clippy commands to AGENTS.md #21171

Merged

-./ci/scripts/rust_fmt.sh
-./ci/scripts/rust_clippy.sh
+./dev/rust_lint.sh
+# use the `--write` flag to automatically fix some formatting and lint errors
+# ./dev/rust_lint.sh --write --allow-dirty

Conversation

alamb commented Mar 16, 2026

Which issue does this PR close?

Rationale for this change

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

adriangb left a comment

Choose a reason for hiding this comment

Uh oh!

2010YOUY01 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

alamb commented Mar 22, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants