Skip to content

Commit e0b3525

Browse files
Add sync/async scrape and extract example parity
Co-authored-by: Shri Sukhani <shrisukhani@users.noreply.github.com>
1 parent 143888b commit e0b3525

5 files changed

Lines changed: 73 additions & 2 deletions

File tree

README.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -258,7 +258,9 @@ Contributor workflow details are available in [CONTRIBUTING.md](CONTRIBUTING.md)
258258
Ready-to-run examples are available in `examples/`:
259259

260260
- `examples/sync_scrape.py`
261+
- `examples/async_scrape.py`
261262
- `examples/async_extract.py`
263+
- `examples/sync_extract.py`
262264
- `examples/sync_web_search.py`
263265
- `examples/async_web_search.py`
264266
- `examples/sync_session_list.py`

examples/async_extract.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@
33
44
Run:
55
export HYPERBROWSER_API_KEY="your_api_key"
6-
python examples/async_extract.py
6+
python3 examples/async_extract.py
77
"""
88

99
import asyncio

examples/async_scrape.py

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
"""
2+
Asynchronous scrape example.
3+
4+
Run:
5+
export HYPERBROWSER_API_KEY="your_api_key"
6+
python3 examples/async_scrape.py
7+
"""
8+
9+
import asyncio
10+
11+
from hyperbrowser import AsyncHyperbrowser
12+
from hyperbrowser.exceptions import HyperbrowserTimeoutError
13+
from hyperbrowser.models import ScrapeOptions, StartScrapeJobParams
14+
15+
16+
async def main() -> None:
17+
async with AsyncHyperbrowser() as client:
18+
try:
19+
result = await client.scrape.start_and_wait(
20+
StartScrapeJobParams(
21+
url="https://hyperbrowser.ai",
22+
scrape_options=ScrapeOptions(formats=["markdown"]),
23+
),
24+
poll_interval_seconds=1.0,
25+
max_wait_seconds=120.0,
26+
)
27+
except HyperbrowserTimeoutError:
28+
print("Scrape job timed out.")
29+
return
30+
31+
if result.data and result.data.markdown:
32+
print(result.data.markdown[:500])
33+
else:
34+
print(
35+
f"Scrape finished with status={result.status} and no markdown payload."
36+
)
37+
38+
39+
if __name__ == "__main__":
40+
asyncio.run(main())

examples/sync_extract.py

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
"""
2+
Synchronous extract example.
3+
4+
Run:
5+
export HYPERBROWSER_API_KEY="your_api_key"
6+
python3 examples/sync_extract.py
7+
"""
8+
9+
from hyperbrowser import Hyperbrowser
10+
from hyperbrowser.models import StartExtractJobParams
11+
12+
13+
def main() -> None:
14+
with Hyperbrowser() as client:
15+
result = client.extract.start_and_wait(
16+
StartExtractJobParams(
17+
urls=["https://hyperbrowser.ai"],
18+
prompt="Extract the main product value propositions as a list.",
19+
),
20+
poll_interval_seconds=1.0,
21+
max_wait_seconds=120.0,
22+
)
23+
24+
print(result.status)
25+
print(result.data)
26+
27+
28+
if __name__ == "__main__":
29+
main()

examples/sync_scrape.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@
33
44
Run:
55
export HYPERBROWSER_API_KEY="your_api_key"
6-
python examples/sync_scrape.py
6+
python3 examples/sync_scrape.py
77
"""
88

99
from hyperbrowser import Hyperbrowser

0 commit comments

Comments
 (0)