Skip to content

New branch for datasegmentation #45

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
JainitBITW wants to merge 6 commits into
base: dev
Choose a base branch
from
Open

New branch for datasegmentation #45

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
bf9c352
Some bugs fixed
JainitBITW Mar 10, 2024
01afa6f
fixed code
hitesh05 Mar 10, 2024
3a0bf50
BaseLine Running
JainitBITW Mar 19, 2024
7a3825d
implemented multifile system to solve the memory issue now training t…
JainitBITW Mar 21, 2024
cf2bc2c
added data splitting script to split the data
JainitBITW Mar 21, 2024
bbe10f0
added data splitting bash script to split
JainitBITW Apr 4, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,6 @@ checkpoints/*
cache/*
save/*
log/*

models/*
# file
*.pyc
4 changes: 4 additions & 0 deletions 0.1.3
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
Requirement already satisfied: dgl in /home2/jainit/miniconda3/envs/mistral/lib/python3.10/site-packages (0.1.3)
Requirement already satisfied: numpy>=1.14.0 in /home2/jainit/miniconda3/envs/mistral/lib/python3.10/site-packages (from dgl) (1.26.3)
Requirement already satisfied: scipy>=1.1.0 in /home2/jainit/miniconda3/envs/mistral/lib/python3.10/site-packages (from dgl) (1.11.4)
Requirement already satisfied: networkx>=2.1 in /home2/jainit/miniconda3/envs/mistral/lib/python3.10/site-packages (from dgl) (3.2.1)
2 changes: 1 addition & 1 deletion HiGraph.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@ def __init__(self, hps, embed):
# sent -> word
self.sent2word = WSWGAT(in_dim=hps.hidden_size,
out_dim=embed_size,
num_heads=6,
num_heads=hps.n_head,
attn_drop_out=hps.atten_dropout_prob,
ffn_inner_hidden_size=hps.ffn_inner_hidden_size,
ffn_drop_out=hps.ffn_dropout_prob,
Expand Down
2 changes: 1 addition & 1 deletion evaluation.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -84,7 +84,7 @@ def run_test(model, dataset, loader, model_name, hps):

for i, (G, index) in enumerate(loader):
if hps.cuda:
G.to(torch.device("cuda"))
G.to(torch.device(0))
tester.evaluation(G, index, dataset, blocking=hps.blocking)

running_avg_loss = tester.running_avg_loss
Expand Down
12 changes: 12 additions & 0 deletions logs/train_20240304_232832
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
2024-03-04 23:28:32,376 INFO : Pytorch 2.1.2+cu121
2024-03-04 23:28:32,376 INFO : [INFO] Create Vocab, vocab path is /scratch/jainit/cache/CNNDM/vocab
2024-03-04 23:28:32,441 INFO : [INFO] max_size of vocab was specified as 50000; we now have 50000 words. Stopping reading.
2024-03-04 23:28:32,442 INFO : [INFO] Finished constructing vocabulary of 50000 total words. Last word added: chaudhary
2024-03-04 23:28:33,288 INFO : [INFO] Loading external word embedding...
2024-03-04 23:28:55,669 INFO : [INFO] External Word Embedding iov count: 48427, oov count: 1573
2024-03-04 23:28:56,428 INFO : Namespace(data_dir='/scratch/jainit/cnndm', cache_dir='/scratch/jainit/cache/CNNDM', embedding_path='/scratch/jainit/glove.6B.300d.txt', model='HSG', restore_model='None', save_root='model', log_root='logs', seed=666, gpu='0', cuda=True, vocab_size=50000, n_epochs=20, batch_size=32, n_iter=1, word_embedding=True, word_emb_dim=300, embed_train=False, feat_embed_size=50, n_layers=1, lstm_hidden_state=128, lstm_layers=2, bidirectional=True, n_feature_size=128, hidden_size=64, ffn_inner_hidden_size=512, n_head=8, recurrent_dropout_prob=0.1, atten_dropout_prob=0.1, ffn_dropout_prob=0.1, use_orthnormal_init=True, sent_max_len=100, doc_max_timesteps=50, lr=0.0005, lr_descent=True, grad_clip=True, max_grad_norm=1.0, m=3)
2024-03-04 23:28:56,521 INFO : [MODEL] HeterSumGraph
2024-03-04 23:28:56,521 INFO : [INFO] Start reading ExampleSet
2024-03-04 23:29:04,974 INFO : [INFO] Finish reading ExampleSet. Total time is 8.453032, Total size is 287084
2024-03-04 23:29:04,975 INFO : [INFO] Loading filter word File /scratch/jainit/cache/CNNDM/filter_word.txt
2024-03-04 23:29:05,195 INFO : [INFO] Loading word2sent TFIDF file from /scratch/jainit/cache/CNNDM/train.w2s.tfidf.jsonl!
12 changes: 12 additions & 0 deletions logs/train_20240304_233425
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
2024-03-04 23:34:25,750 INFO : Pytorch 2.1.2+cu121
2024-03-04 23:34:25,750 INFO : [INFO] Create Vocab, vocab path is /scratch/jainit/cache/CNNDM/vocab
2024-03-04 23:34:25,820 INFO : [INFO] max_size of vocab was specified as 50000; we now have 50000 words. Stopping reading.
2024-03-04 23:34:25,820 INFO : [INFO] Finished constructing vocabulary of 50000 total words. Last word added: chaudhary
2024-03-04 23:34:26,728 INFO : [INFO] Loading external word embedding...
2024-03-04 23:34:48,983 INFO : [INFO] External Word Embedding iov count: 48427, oov count: 1573
2024-03-04 23:34:49,800 INFO : Namespace(data_dir='/scratch/jainit/cnndm', cache_dir='/scratch/jainit/cache/CNNDM', embedding_path='/scratch/jainit/glove.6B.300d.txt', model='HSG', restore_model='None', save_root='model', log_root='logs', seed=666, gpu='0', cuda=True, vocab_size=50000, n_epochs=20, batch_size=32, n_iter=1, word_embedding=True, word_emb_dim=300, embed_train=False, feat_embed_size=50, n_layers=1, lstm_hidden_state=128, lstm_layers=2, bidirectional=True, n_feature_size=128, hidden_size=64, ffn_inner_hidden_size=512, n_head=8, recurrent_dropout_prob=0.1, atten_dropout_prob=0.1, ffn_dropout_prob=0.1, use_orthnormal_init=True, sent_max_len=100, doc_max_timesteps=50, lr=0.0005, lr_descent=True, grad_clip=True, max_grad_norm=1.0, m=3)
2024-03-04 23:34:49,894 INFO : [MODEL] HeterSumGraph
2024-03-04 23:34:49,894 INFO : [INFO] Start reading ExampleSet
2024-03-04 23:34:58,247 INFO : [INFO] Finish reading ExampleSet. Total time is 8.353097, Total size is 287084
2024-03-04 23:34:58,247 INFO : [INFO] Loading filter word File /scratch/jainit/cache/CNNDM/filter_word.txt
2024-03-04 23:34:58,468 INFO : [INFO] Loading word2sent TFIDF file from /scratch/jainit/cache/CNNDM/train.w2s.tfidf.jsonl!
12 changes: 12 additions & 0 deletions logs/train_20240304_233757
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
2024-03-04 23:37:57,657 INFO : Pytorch 2.1.2+cu121
2024-03-04 23:37:57,657 INFO : [INFO] Create Vocab, vocab path is /scratch/jainit/cache/CNNDM/vocab
2024-03-04 23:37:57,709 INFO : [INFO] max_size of vocab was specified as 50000; we now have 50000 words. Stopping reading.
2024-03-04 23:37:57,709 INFO : [INFO] Finished constructing vocabulary of 50000 total words. Last word added: chaudhary
2024-03-04 23:37:58,621 INFO : [INFO] Loading external word embedding...
2024-03-04 23:38:20,757 INFO : [INFO] External Word Embedding iov count: 48427, oov count: 1573
2024-03-04 23:38:21,573 INFO : Namespace(data_dir='/scratch/jainit/cnndm', cache_dir='/scratch/jainit/cache/CNNDM', embedding_path='/scratch/jainit/glove.6B.300d.txt', model='HSG', restore_model='None', save_root='model', log_root='logs', seed=666, gpu='0', cuda=True, vocab_size=50000, n_epochs=20, batch_size=32, n_iter=1, word_embedding=True, word_emb_dim=300, embed_train=False, feat_embed_size=50, n_layers=1, lstm_hidden_state=128, lstm_layers=2, bidirectional=True, n_feature_size=128, hidden_size=64, ffn_inner_hidden_size=512, n_head=8, recurrent_dropout_prob=0.1, atten_dropout_prob=0.1, ffn_dropout_prob=0.1, use_orthnormal_init=True, sent_max_len=100, doc_max_timesteps=50, lr=0.0005, lr_descent=True, grad_clip=True, max_grad_norm=1.0, m=3)
2024-03-04 23:38:21,676 INFO : [MODEL] HeterSumGraph
2024-03-04 23:38:21,676 INFO : [INFO] Start reading ExampleSet
2024-03-04 23:38:30,203 INFO : [INFO] Finish reading ExampleSet. Total time is 8.526127, Total size is 287084
2024-03-04 23:38:30,203 INFO : [INFO] Loading filter word File /scratch/jainit/cache/CNNDM/filter_word.txt
2024-03-04 23:38:30,435 INFO : [INFO] Loading word2sent TFIDF file from /scratch/jainit/cache/CNNDM/train.w2s.tfidf.jsonl!
19 changes: 19 additions & 0 deletions logs/train_20240307_224013
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
2024-03-07 22:40:13,242 INFO : Pytorch 2.1.2+cu121
2024-03-07 22:40:13,242 INFO : [INFO] Create Vocab, vocab path is /scratch/jainit/cache/CNNDM/vocab
2024-03-07 22:40:13,313 INFO : [INFO] max_size of vocab was specified as 50000; we now have 50000 words. Stopping reading.
2024-03-07 22:40:13,313 INFO : [INFO] Finished constructing vocabulary of 50000 total words. Last word added: chaudhary
2024-03-07 22:40:13,619 INFO : [INFO] Loading external word embedding...
2024-03-07 22:40:32,029 INFO : [INFO] External Word Embedding iov count: 48427, oov count: 1573
2024-03-07 22:40:32,809 INFO : Namespace(data_dir='/scratch/jainit/cnndm', cache_dir='/scratch/jainit/cache/CNNDM', embedding_path='/scratch/jainit/glove.6B.300d.txt', model='HSG', restore_model='None', save_root='models', log_root='logs', seed=666, gpu='0', cuda=True, vocab_size=50000, n_epochs=20, batch_size=32, n_iter=1, word_embedding=True, word_emb_dim=300, embed_train=False, feat_embed_size=50, n_layers=1, lstm_hidden_state=128, lstm_layers=2, bidirectional=True, n_feature_size=128, hidden_size=64, ffn_inner_hidden_size=512, n_head=8, recurrent_dropout_prob=0.1, atten_dropout_prob=0.1, ffn_dropout_prob=0.1, use_orthnormal_init=True, sent_max_len=100, doc_max_timesteps=50, lr=0.0005, lr_descent=True, grad_clip=True, max_grad_norm=1.0, m=3)
2024-03-07 22:40:32,906 INFO : [MODEL] HeterSumGraph
2024-03-07 22:40:32,906 INFO : [INFO] Start reading ExampleSet
2024-03-07 22:40:39,757 INFO : [INFO] Finish reading ExampleSet. Total time is 6.850199, Total size is 287084
2024-03-07 22:40:39,757 INFO : [INFO] Loading filter word File /scratch/jainit/cache/CNNDM/filter_word.txt
2024-03-07 22:40:39,965 INFO : [INFO] Loading word2sent TFIDF file from /scratch/jainit/cache/CNNDM/train.w2s.tfidf.jsonl!
2024-03-07 22:42:27,874 INFO : [INFO] Start reading ExampleSet
2024-03-07 22:42:28,182 INFO : [INFO] Finish reading ExampleSet. Total time is 0.307474, Total size is 13367
2024-03-07 22:42:28,182 INFO : [INFO] Loading filter word File /scratch/jainit/cache/CNNDM/filter_word.txt
2024-03-07 22:42:28,436 INFO : [INFO] Loading word2sent TFIDF file from /scratch/jainit/cache/CNNDM/val.w2s.tfidf.jsonl!
2024-03-07 22:42:41,313 INFO : [INFO] Use cuda
2024-03-07 22:42:41,314 INFO : [INFO] Create new model for training...
2024-03-07 22:42:41,314 INFO : [INFO] Starting run_training
19 changes: 19 additions & 0 deletions logs/train_20240307_224949
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
2024-03-07 22:49:49,256 INFO : Pytorch 2.1.2+cu121
2024-03-07 22:49:49,256 INFO : [INFO] Create Vocab, vocab path is /scratch/jainit/cache/CNNDM/vocab
2024-03-07 22:49:49,322 INFO : [INFO] max_size of vocab was specified as 50000; we now have 50000 words. Stopping reading.
2024-03-07 22:49:49,322 INFO : [INFO] Finished constructing vocabulary of 50000 total words. Last word added: chaudhary
2024-03-07 22:49:49,519 INFO : [INFO] Loading external word embedding...
2024-03-07 22:50:07,671 INFO : [INFO] External Word Embedding iov count: 48427, oov count: 1573
2024-03-07 22:50:08,462 INFO : Namespace(data_dir='/scratch/jainit/cnndm', cache_dir='/scratch/jainit/cache/CNNDM', embedding_path='/scratch/jainit/glove.6B.300d.txt', model='HSG', restore_model='None', save_root='models', log_root='logs', seed=666, gpu='0', cuda=True, vocab_size=50000, n_epochs=20, batch_size=32, n_iter=1, word_embedding=True, word_emb_dim=300, embed_train=False, feat_embed_size=50, n_layers=1, lstm_hidden_state=128, lstm_layers=2, bidirectional=True, n_feature_size=128, hidden_size=64, ffn_inner_hidden_size=512, n_head=8, recurrent_dropout_prob=0.1, atten_dropout_prob=0.1, ffn_dropout_prob=0.1, use_orthnormal_init=True, sent_max_len=100, doc_max_timesteps=50, lr=0.0005, lr_descent=True, grad_clip=True, max_grad_norm=1.0, m=3)
2024-03-07 22:50:08,564 INFO : [MODEL] HeterSumGraph
2024-03-07 22:50:08,565 INFO : [INFO] Start reading ExampleSet
2024-03-07 22:50:15,644 INFO : [INFO] Finish reading ExampleSet. Total time is 7.079171, Total size is 287084
2024-03-07 22:50:15,644 INFO : [INFO] Loading filter word File /scratch/jainit/cache/CNNDM/filter_word.txt
2024-03-07 22:50:15,854 INFO : [INFO] Loading word2sent TFIDF file from /scratch/jainit/cache/CNNDM/train.w2s.tfidf.jsonl!
2024-03-07 22:51:59,319 INFO : [INFO] Start reading ExampleSet
2024-03-07 22:51:59,541 INFO : [INFO] Finish reading ExampleSet. Total time is 0.220881, Total size is 13367
2024-03-07 22:51:59,541 INFO : [INFO] Loading filter word File /scratch/jainit/cache/CNNDM/filter_word.txt
2024-03-07 22:51:59,754 INFO : [INFO] Loading word2sent TFIDF file from /scratch/jainit/cache/CNNDM/val.w2s.tfidf.jsonl!
2024-03-07 22:52:09,072 INFO : [INFO] Use cuda
2024-03-07 22:52:09,073 INFO : [INFO] Create new model for training...
2024-03-07 22:52:09,073 INFO : [INFO] Starting run_training
19 changes: 19 additions & 0 deletions logs/train_20240307_230618
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
2024-03-07 23:06:18,345 INFO : Pytorch 2.1.2+cu121
2024-03-07 23:06:18,346 INFO : [INFO] Create Vocab, vocab path is /scratch/jainit/cache/CNNDM/vocab
2024-03-07 23:06:18,412 INFO : [INFO] max_size of vocab was specified as 50000; we now have 50000 words. Stopping reading.
2024-03-07 23:06:18,412 INFO : [INFO] Finished constructing vocabulary of 50000 total words. Last word added: chaudhary
2024-03-07 23:06:18,591 INFO : [INFO] Loading external word embedding...
2024-03-07 23:06:37,131 INFO : [INFO] External Word Embedding iov count: 48427, oov count: 1573
2024-03-07 23:06:37,935 INFO : Namespace(data_dir='/scratch/jainit/cnndm', cache_dir='/scratch/jainit/cache/CNNDM', embedding_path='/scratch/jainit/glove.6B.300d.txt', model='HSG', restore_model='None', save_root='models', log_root='logs', seed=666, gpu='0', cuda=True, vocab_size=50000, n_epochs=20, batch_size=32, n_iter=1, word_embedding=True, word_emb_dim=300, embed_train=False, feat_embed_size=50, n_layers=1, lstm_hidden_state=128, lstm_layers=2, bidirectional=True, n_feature_size=128, hidden_size=64, ffn_inner_hidden_size=512, n_head=8, recurrent_dropout_prob=0.1, atten_dropout_prob=0.1, ffn_dropout_prob=0.1, use_orthnormal_init=True, sent_max_len=100, doc_max_timesteps=50, lr=0.0005, lr_descent=True, grad_clip=True, max_grad_norm=1.0, m=3)
2024-03-07 23:06:38,036 INFO : [MODEL] HeterSumGraph
2024-03-07 23:06:38,037 INFO : [INFO] Start reading ExampleSet
2024-03-07 23:06:45,035 INFO : [INFO] Finish reading ExampleSet. Total time is 6.998097, Total size is 287084
2024-03-07 23:06:45,035 INFO : [INFO] Loading filter word File /scratch/jainit/cache/CNNDM/filter_word.txt
2024-03-07 23:06:45,273 INFO : [INFO] Loading word2sent TFIDF file from /scratch/jainit/cache/CNNDM/train.w2s.tfidf.jsonl!
2024-03-07 23:08:30,094 INFO : [INFO] Start reading ExampleSet
2024-03-07 23:08:30,322 INFO : [INFO] Finish reading ExampleSet. Total time is 0.227785, Total size is 13367
2024-03-07 23:08:30,322 INFO : [INFO] Loading filter word File /scratch/jainit/cache/CNNDM/filter_word.txt
2024-03-07 23:08:30,548 INFO : [INFO] Loading word2sent TFIDF file from /scratch/jainit/cache/CNNDM/val.w2s.tfidf.jsonl!
2024-03-07 23:08:37,672 INFO : [INFO] Use cuda
2024-03-07 23:08:37,673 INFO : [INFO] Create new model for training...
2024-03-07 23:08:37,673 INFO : [INFO] Starting run_training
19 changes: 19 additions & 0 deletions logs/train_20240307_231149
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
2024-03-07 23:11:49,495 INFO : Pytorch 2.1.2+cu121
2024-03-07 23:11:49,496 INFO : [INFO] Create Vocab, vocab path is /scratch/jainit/cache/CNNDM/vocab
2024-03-07 23:11:49,567 INFO : [INFO] max_size of vocab was specified as 50000; we now have 50000 words. Stopping reading.
2024-03-07 23:11:49,567 INFO : [INFO] Finished constructing vocabulary of 50000 total words. Last word added: chaudhary
2024-03-07 23:11:49,724 INFO : [INFO] Loading external word embedding...
2024-03-07 23:12:08,091 INFO : [INFO] External Word Embedding iov count: 48427, oov count: 1573
2024-03-07 23:12:08,910 INFO : Namespace(data_dir='/scratch/jainit/cnndm', cache_dir='/scratch/jainit/cache/CNNDM', embedding_path='/scratch/jainit/glove.6B.300d.txt', model='HSG', restore_model='None', save_root='models', log_root='logs', seed=666, gpu='0', cuda=True, vocab_size=50000, n_epochs=20, batch_size=32, n_iter=1, word_embedding=True, word_emb_dim=300, embed_train=False, feat_embed_size=50, n_layers=1, lstm_hidden_state=128, lstm_layers=2, bidirectional=True, n_feature_size=128, hidden_size=64, ffn_inner_hidden_size=512, n_head=8, recurrent_dropout_prob=0.1, atten_dropout_prob=0.1, ffn_dropout_prob=0.1, use_orthnormal_init=True, sent_max_len=100, doc_max_timesteps=50, lr=0.0005, lr_descent=True, grad_clip=True, max_grad_norm=1.0, m=3)
2024-03-07 23:12:09,064 INFO : [MODEL] HeterSumGraph
2024-03-07 23:12:09,064 INFO : [INFO] Start reading ExampleSet
2024-03-07 23:12:17,707 INFO : [INFO] Finish reading ExampleSet. Total time is 8.642636, Total size is 287084
2024-03-07 23:12:17,707 INFO : [INFO] Loading filter word File /scratch/jainit/cache/CNNDM/filter_word.txt
2024-03-07 23:12:17,932 INFO : [INFO] Loading word2sent TFIDF file from /scratch/jainit/cache/CNNDM/train.w2s.tfidf.jsonl!
2024-03-07 23:14:02,867 INFO : [INFO] Start reading ExampleSet
2024-03-07 23:14:05,248 INFO : [INFO] Finish reading ExampleSet. Total time is 2.381086, Total size is 13367
2024-03-07 23:14:05,248 INFO : [INFO] Loading filter word File /scratch/jainit/cache/CNNDM/filter_word.txt
2024-03-07 23:14:05,481 INFO : [INFO] Loading word2sent TFIDF file from /scratch/jainit/cache/CNNDM/val.w2s.tfidf.jsonl!
2024-03-07 23:14:12,124 INFO : [INFO] Use cuda
2024-03-07 23:14:12,124 INFO : [INFO] Create new model for training...
2024-03-07 23:14:12,125 INFO : [INFO] Starting run_training
19 changes: 19 additions & 0 deletions logs/train_20240307_235940
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
2024-03-07 23:59:40,422 INFO : Pytorch 2.1.2+cu121
2024-03-07 23:59:40,423 INFO : [INFO] Create Vocab, vocab path is /scratch/jainit/cache/CNNDM/vocab
2024-03-07 23:59:40,479 INFO : [INFO] max_size of vocab was specified as 50000; we now have 50000 words. Stopping reading.
2024-03-07 23:59:40,479 INFO : [INFO] Finished constructing vocabulary of 50000 total words. Last word added: chaudhary
2024-03-07 23:59:40,631 INFO : [INFO] Loading external word embedding...
2024-03-08 00:00:03,782 INFO : [INFO] External Word Embedding iov count: 48427, oov count: 1573
2024-03-08 00:00:04,691 INFO : Namespace(data_dir='/scratch/jainit/cnndm', cache_dir='/scratch/jainit/cache/CNNDM', embedding_path='/scratch/jainit/glove.6B.300d.txt', model='HSG', restore_model='None', save_root='models', log_root='logs', seed=666, gpu='0', cuda=True, vocab_size=50000, n_epochs=20, batch_size=32, n_iter=1, word_embedding=True, word_emb_dim=300, embed_train=False, feat_embed_size=50, n_layers=1, lstm_hidden_state=128, lstm_layers=2, bidirectional=True, n_feature_size=128, hidden_size=64, ffn_inner_hidden_size=512, n_head=8, recurrent_dropout_prob=0.1, atten_dropout_prob=0.1, ffn_dropout_prob=0.1, use_orthnormal_init=True, sent_max_len=100, doc_max_timesteps=50, lr=0.0005, lr_descent=True, grad_clip=True, max_grad_norm=1.0, m=3)
2024-03-08 00:00:04,796 INFO : [MODEL] HeterSumGraph
2024-03-08 00:00:04,797 INFO : [INFO] Start reading ExampleSet
2024-03-08 00:00:13,417 INFO : [INFO] Finish reading ExampleSet. Total time is 8.620221, Total size is 287084
2024-03-08 00:00:13,417 INFO : [INFO] Loading filter word File /scratch/jainit/cache/CNNDM/filter_word.txt
2024-03-08 00:00:13,633 INFO : [INFO] Loading word2sent TFIDF file from /scratch/jainit/cache/CNNDM/train.w2s.tfidf.jsonl!
2024-03-08 00:01:58,838 INFO : [INFO] Start reading ExampleSet
2024-03-08 00:02:02,842 INFO : [INFO] Finish reading ExampleSet. Total time is 4.002773, Total size is 13367
2024-03-08 00:02:02,842 INFO : [INFO] Loading filter word File /scratch/jainit/cache/CNNDM/filter_word.txt
2024-03-08 00:02:03,065 INFO : [INFO] Loading word2sent TFIDF file from /scratch/jainit/cache/CNNDM/val.w2s.tfidf.jsonl!
2024-03-08 00:02:09,908 INFO : [INFO] Use cuda
2024-03-08 00:02:09,908 INFO : [INFO] Create new model for training...
2024-03-08 00:02:09,908 INFO : [INFO] Starting run_training
Loading