Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
110 commits
Select commit Hold shift + click to select a range
2ef5919
Enhance deferred fields for production ETL reliability
bosd Dec 21, 2025
e08aa84
Fix CLI deferred-fields parsing and Pass 1 ignore filtering
bosd Dec 21, 2025
d10b5f3
feat: add --auto-defer option for progressive import mode
bosd Dec 21, 2025
7ec5c72
fix: create ir.model.data entries when using create() method
bosd Dec 21, 2025
583ed63
feat: add advanced import options for handling missing refs and fallb…
bosd Dec 22, 2025
bd461fa
perf: remove connection cap, add caching and pre-calculation
bosd Dec 22, 2025
a277b51
feat: add --protocol option for RPC protocol selection
bosd Dec 22, 2025
d1599d1
fix: convert --ignore from comma-separated string to list
bosd Dec 22, 2025
026aa75
docs: add protocol selection and worker tuning documentation
bosd Dec 22, 2025
8314992
test: add Unicode and multiline CSV handling test
bosd Dec 22, 2025
5a3be70
feat: add --delay option for rate limiting between batches
bosd Dec 22, 2025
b9e4c18
feat: add adaptive throttling for 503/502 server overload errors
bosd Dec 22, 2025
47d6278
fix: prevent Rich progress bar shifting by suppressing log handler
bosd Dec 22, 2025
07ef183
chore: update mypyc configuration for better compatibility
bosd Dec 22, 2025
6fa9cc5
test: improve test coverage to 85%
bosd Dec 22, 2025
797d68a
feat: add streaming CSV support for memory-efficient large file imports
bosd Dec 23, 2025
fedcd43
Add checkpoint/resume support and --all-companies flag
bosd Dec 23, 2025
0a5e7b2
feat: add dry-run validation mode
bosd Dec 23, 2025
061e6eb
feat: add pre-import reference check
bosd Dec 23, 2025
f57cc02
feat: add smart retry logic module
bosd Dec 23, 2025
442f61c
feat: add idempotent import module
bosd Dec 23, 2025
9896813
feat: add health-aware throttling module
bosd Dec 23, 2025
d3acf0c
feat: integrate retry, idempotent, and throttle modules
bosd Dec 24, 2025
86bce87
Add VIES/VAT validation management workflow
bosd Dec 25, 2025
00341ea
Add documentation for VAT validation management
bosd Dec 25, 2025
2f8b036
Fix linting and type annotations for pre-commit/mypy compliance
bosd Dec 26, 2025
e5942b0
Fix mypy type errors in validation, idempotent, and preflight modules
bosd Dec 26, 2025
a228570
Add date_formats and datetime_formats parameters to Processor
bosd Dec 26, 2025
a478535
Add expr module with Polars expression-based mappers
bosd Dec 26, 2025
1f741c6
Add type annotations to test files for mypy compliance
bosd Dec 26, 2025
d0913e4
Integrate dynamic batch size scaling into import process
bosd Dec 27, 2025
10d8032
Add adaptive throttling documentation and fix test type annotations
bosd Dec 27, 2025
a83db51
Add data cleaning module for transformation pipelines
bosd Dec 28, 2025
59d001c
Add data cleaning documentation to transformation guide
bosd Dec 29, 2025
60e6b29
Improve phone normalization to detect country codes and 00 prefix
bosd Dec 29, 2025
bcf0d9f
Add email cleaner support for colons (mailto: prefix, separators, tra…
bosd Dec 29, 2025
486a624
feat(clean): add city/postal separator and country detection
bosd Dec 29, 2025
ed5a969
feat(clean): add company_suffix() for business entity normalization
bosd Dec 29, 2025
3303e14
refactor(clean): remove hardcoded MAJOR_CITIES constant
bosd Dec 29, 2025
b3fce15
docs: add company_suffix() examples to data transformations guide
bosd Dec 29, 2025
0326375
feat(geonames): add GeoNames data utilities for geographic lookups
bosd Dec 29, 2025
ea439b5
fix(clean): improve zip_code() to filter e- prefix and remove commas
bosd Dec 29, 2025
59ae2e5
docs: update zip_code() documentation with new features
bosd Dec 29, 2025
84a1afe
feat(clean): add city() cleaner for city name normalization
bosd Dec 29, 2025
223acf2
feat(clean): add street() cleaner for street address cleaning
bosd Dec 29, 2025
fcb4bc5
fix(preflight): exclude 'id' field from readonly warning
bosd Dec 31, 2025
9cfb151
feat: environment-based fail file placement
bosd Dec 31, 2025
a274b31
docs: add multi-environment imports section
bosd Dec 31, 2025
10594f2
feat: improve access error messages in fail files
bosd Dec 31, 2025
94cc664
fix: avoid using model.browse() for models where it's not allowed rem…
bosd Jan 2, 2026
47ec5e4
fix: remove remaining model.env.ref() usage that triggered browse errors
bosd Jan 2, 2026
70458cf
fix: avoid triggering browse when accessing record ID from create()
bosd Jan 2, 2026
24ed6be
fix: sanitize id_map lookups in pass 2 for parent_id resolution
bosd Jan 4, 2026
4bf04a3
fix: add logging around thread pool shutdown to diagnose hanging
bosd Jan 4, 2026
0b6e393
fix: use console.print for diagnostic messages during progress display
bosd Jan 4, 2026
de1587e
fix: add diagnostic logging throughout Pass 2
bosd Jan 4, 2026
1471f1a
fix: add granular diagnostic logging inside _prepare_pass_2_data
bosd Jan 4, 2026
994f892
fix: add counters to track id_map hits vs RPC lookups in Pass 2
bosd Jan 4, 2026
220b62d
perf: add caching and better progress for Pass 2 external ID lookups
bosd Jan 4, 2026
9e8efd9
fix: prevent automatic deferral of m2m fields without user consent
bosd Jan 5, 2026
26591af
fix: suppress confusing deferrable fields message when not auto-defer…
bosd Jan 5, 2026
b638e49
refactor: use load() instead of create() for fallback imports
bosd Jan 5, 2026
1b85b13
fix: filter out empty strings from required languages check
bosd Jan 10, 2026
b739edd
feat: add --all-companies flag to export command
bosd Jan 11, 2026
9a7bcc6
fix: add domain filter for export --all-companies
bosd Jan 11, 2026
2baaf1d
feat: add --sudo flag to export for bypassing record rules
bosd Jan 11, 2026
95306b3
fix: check for company_id field before adding domain filter
bosd Jan 11, 2026
ce2d100
fix: improve error reporting for failed export records
bosd Jan 11, 2026
ec32e54
fix: sudo mode now disables rules for related models too
bosd Jan 11, 2026
3a58dc5
feat: auto-generate XML IDs for rows with empty id values
bosd Jan 13, 2026
7ad4ffc
fix: ensure XML IDs are persisted after load() succeeds
bosd Jan 16, 2026
00514b2
docs: fix misleading 'create method' messages and documentation
bosd Jan 16, 2026
5671b6f
perf: optimize Pass 2 batching to reduce RPC overhead
bosd Jan 17, 2026
66423bd
perf: use binary search fallback for batch import failures
bosd Jan 17, 2026
94d6922
fix: exclude self-references from missing reference warnings
bosd Jan 17, 2026
9f20daf
feat: add file-based backup for VAT validation settings
bosd Jan 17, 2026
6e374c7
feat: add --post-action flag to execute methods on imported records
bosd Jan 18, 2026
65d6fc0
feat: add --skip-existing flag for idempotent imports
bosd Jan 18, 2026
d682dba
docs: add guide for importing company-dependent cost prices
bosd Jan 20, 2026
bbb6865
feat: auto-detect company-dependent fields in preflight checks
bosd Jan 21, 2026
1e7dce8
feat: allow --company-id to accept XML IDs
bosd Jan 21, 2026
2d5c08a
feat: add --move-date flag for opening inventory imports
bosd Jan 21, 2026
c2df94b
fix: improve --move-date reliability for production use
bosd Jan 21, 2026
a1eeca5
fix: prevent polars type inference errors on mixed-type columns
bosd Jan 21, 2026
378dc00
feat: improve server crash recovery for remote imports
bosd Jan 22, 2026
d4e021b
feat: enable adaptive throttle by default for stability
bosd Jan 22, 2026
6257a15
fix: resolve lint and type errors in CI
bosd Jan 23, 2026
571e559
fix: use configparser instead of yaml for config file parsing
bosd Jan 23, 2026
0cf4551
fix: resolve mypy errors in importer.py
bosd Jan 23, 2026
e31e7ab
test: improve test coverage from 81% to 85%
bosd Jan 24, 2026
608f1bf
fix: add UTF-8 encoding to test file writes for Windows compatibility
bosd Jan 24, 2026
6dd72cc
fix: add extra context flags to fully suppress mail tracking
bosd Jan 29, 2026
bae0f0d
fix: run post-action even after partial import
bosd Feb 3, 2026
20f27b5
fix: many2many /.id export now returns all IDs instead of only the first
bosd Feb 11, 2026
ed830f2
fix: correctly handle many2one vs many2many in /.id export
Feb 11, 2026
33fd8b6
fix: many2many/one2many /id export now returns all related XML IDs
bosd Feb 11, 2026
3089cb6
fix: remove deprecated force_company context key for Odoo 18+
bosd Feb 12, 2026
bd337f3
Add size-based batching for large payload imports
Feb 13, 2026
a321226
fix: resolve lint issues and add max_batch_bytes parameter to streaming
bosd Feb 13, 2026
f1249d0
fix: resolve mypy and typeguard issues
bosd Feb 14, 2026
d68f7a0
fix: handle many2many fields correctly in Pass 2 deferred updates
Feb 14, 2026
034ea39
fix: handle nested lists in Pass 2 grouping for many2many fields
Feb 14, 2026
65252f2
fix: track silently dropped records during import (#178)
bosd Feb 28, 2026
d222f66
fix: add type annotation for update_vals in Pass 2
bosd Feb 28, 2026
6341e4a
fix: resolve cross-model XML IDs in Pass 2 deferred fields (#179)
bosd Feb 28, 2026
888b646
test: add unit tests for cross-model XML ID resolution (#179)
bosd Feb 28, 2026
81d0303
fix: address issues #180, #181, #182
bosd Mar 1, 2026
a2c4a6d
fix: datetime fields exported as empty due to Polars cast failure (#184)
bosd Mar 2, 2026
84ee5e4
fix: add optional newline sanitization for text fields during export …
bosd Mar 3, 2026
75c3129
fix: resolve mypy type annotation errors and code formatting
bosd Mar 3, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -60,3 +60,7 @@ node_modules
_build
ODF_ Strategic Blueprint.md
.coverage

# mypyc compiled extensions
*.so
build/
33 changes: 33 additions & 0 deletions docs/faq.md
Original file line number Diff line number Diff line change
Expand Up @@ -248,3 +248,36 @@ To check for the second case, look at the console output when you run the export

`WARNING Field 'your_field_name' (base: 'your_field_name') not found on model 'res.partner'. An empty column will be created.`
If you see this warning, correct the field name in your command and run the export again.

## VAT validation is stuck in "disabled" state after an import

When importing contact data, the importer temporarily disables VAT validation (VIES checks) to prevent timeouts. If the restoration fails (e.g., due to a 503 error), the settings may remain disabled.

**Symptoms:**
- VIES VAT validation no longer runs when saving contacts
- You see a backup file at `~/.odoo-data-flow/vat_settings_backup/`

**Solution:**

The importer uses a file-based backup system to preserve original settings. You can manually restore them:

```python
from odoo_data_flow.lib.actions.vies_manager import restore_vat_settings_from_backup

success = restore_vat_settings_from_backup("conf/connection.conf")
if success:
print("Settings restored!")
```

Or check the backup status first:

```python
from odoo_data_flow.lib.actions.vies_manager import check_vat_settings_backup_status

status = check_vat_settings_backup_status("conf/connection.conf")
print(f"Backup exists: {status['exists']}")
if status['exists']:
print(f"Age: {status['age_hours']:.1f} hours")
```

> For more details, see [VAT Validation Settings Recovery](guides/advanced_usage.md#vat-validation-settings-recovery) in the Advanced Usage guide.
Loading
Loading