Skip to content

The log file was quickly filled up. #184

@AlbertLin0

Description

@AlbertLin0

Describe the bug
there are a lot of same output in the console:
(KVStorageActor pid=6738) RocksDB flushed for partition
(KVStorageActor pid=6738) RocksDB flushed for partition
(KVStorageActor pid=6738) RocksDB flushed for partition
(KVStorageActor pid=6738) RocksDB flushed for partition
(KVStorageActor pid=6738) RocksDB flushed for partition
(KVStorageActor pid=6738) RocksDB flushed for partition
(KVStorageActor pid=6738) RocksDB flushed for partition
(KVStorageActor pid=6738) RocksDB flushed for partition
(KVStorageActor pid=6738) RocksDB flushed for partition
(KVStorageActor pid=6738) RocksDB flushed for partition
(KVStorageActor pid=6738) RocksDB flushed for partition
(KVStorageActor pid=6738) RocksDB flushed for partition
(KVStorageActor pid=6738) RocksDB flushed for partition
(KVStorageActor pid=6738) RocksDB flushed for partition
(KVStorageActor pid=6738) RocksDB flushed for partition
(KVStorageActor pid=6738) RocksDB flushed for partition
(KVStorageActor pid=6738) RocksDB flushed for partition

To Reproduce
atomic_config.yaml:

global_params:
  working_dir: cache
  graph_backend: kuzu # graph database backend, support: kuzu, networkx
  kv_backend: rocksdb # key-value store backend, support: rocksdb, json_kv

nodes:
  - id: read
    op_name: read
    type: source
    dependencies: []
    params:
      input_path:
        - examples/input_examples/red_yellow_demo.pdf

  - id: chunk
    op_name: chunk
    type: map_batch
    dependencies:
      - read
    execution_params:
      replicas: 4
    params:
      chunk_size: 1024
      chunk_overlap: 100

  - id: build_kg
    op_name: build_kg
    type: map_batch
    execution_params:
      replicas: 1
      batch_size: 8
    dependencies:
      - chunk

  - id: partition
    op_name: partition
    type: aggregate
    dependencies:
      - build_kg
    params:
      method: dfs
      method_params:
        max_units_per_community: 1

  - id: generate
    op_name: generate
    type: map_batch
    dependencies:
      - partition
    execution_params:
      replicas: 1
      batch_size: 8
    save_output: true
    params:
      method: atomic
      data_format: Alpaca

and I config mineru with http format.

Metadata

Metadata

Labels

questionFurther information is requested

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions