Journal of Data Science logo


Login Register

  1. Home
  2. Issues
  3. Volume 19, Issue 3 (2021)
  4. Augmented Abstractive Summarization with ...

Journal of Data Science

Submit your article Information
  • Article info
  • More
    Article info

Augmented Abstractive Summarization with Document-Level Semantic Graph
Volume 19, Issue 3 (2021), pp. 450–464
Qiwei Bi   Haoyuan Li   Kun Lu     All authors (4)

Authors

 
Placeholder
https://doi.org/10.6339/21-JDS1012
Pub. online: 4 May 2021      Type: Data Science In Action     

Received
7 December 2020
Accepted
10 April 2021
Published
4 May 2021

Abstract

Previous abstractive methods apply sequence-to-sequence structures to generate summary without a module to assist the system to detect vital mentions and relationships within a document. To address this problem, we utilize semantic graph to boost the generation performance. Firstly, we extract important entities from each document and then establish a graph inspired by the idea of distant supervision (Mintz et al., 2009). Then, we combine a Bi-LSTM with a graph encoder to obtain the representation of each graph node. A novel neural decoder is presented to leverage the information of such entity graphs. Automatic and human evaluations show the effectiveness of our technique.

Supplementary material

 Supplementary Material
Supplementary material online include: data link, python code and an instruction file needed to reproduce the results; an appendix containing additional structures and experiments we have tried. The web link is https://github.com/martin6336/DSGSum.

References

 
Bahdanau D, Cho K, Bengio Y (2014). Neural machine translation by jointly learning to align and translate. arXiv preprint: https://arxiv.org/abs/1409.0473.
 
Banarescu L, Bonial C, Cai S, Georgescu M, Griffitt K, Hermjakob U, et al. (2013). Abstract Meaning Representation for sembanking. In: Proceedings of the 7th Linguistic Annotation Workshop and Interoperability with Discourse (S Dipper, M Liakata, A Pareja-Lora, eds.), 178–186. Association for Computational Linguistics, Sofia, Bulgaria.
 
Berg-Kirkpatrick T, Burkett D, Klein D (2012). An empirical investigation of statistical significance in NLP. In: Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (M Pasca, J Henderson, J Tsujii, eds.), 995–1005. Association for Computational Linguistics, Jeju Island, Korea.
 
Celikyilmaz A, Bosselut A, He X, Choi Y (2018). Deep communicating agents for abstractive summarization. In: Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) (A Stent, H Ji, MA Walker, eds.), 1662–1675. Association for Computational Linguistics, New Orleans, Louisiana.
 
Chen YC, Bansal M (2018). Fast abstractive summarization with reinforce-selected sentence rewriting. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (Y Miyao, I Gurevych, eds.), 675–686. Association for Computational Linguistics, Melbourne, Australia.
 
Damonte M, Cohen SB (2019). Structural neural encoders for AMR-to-text generation. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) (T Solorio, C Doran, J Burstein, eds.), 3649–3658. Association for Computational Linguistics, Minneapolis, Minnesota.
 
Durrett G, Berg-Kirkpatrick T, Klein D (2016). Learning-based single-document summarization with compression and anaphoricity constraints. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 1998–2008. Association for Computational Linguistics, Berlin, Germany.
 
Fan L, Yu D, Wang L (2018). Robust neural abstractive summarization systems and evaluation against adversarial information. arXiv preprint: https://arxiv.org/abs/1810.06065
 
Fernandes P, Allamanis M, Brockschmidt M (2019). Structured neural summarization.
 
Gehrmann S, Deng Y, Rush A (2018). Bottom-up abstractive summarization. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing (J Tsujii, J Hockenmaier, D Chiang, E Riloff, eds.), 4098–4109. Association for Computational Linguistics, Brussels, Belgium.
 
Guu K, Lee K, Tung Z, Pasupat P, Chang MW (2020). Realm: Retrieval-augmented language model pre-training. arXiv preprint: https://arxiv.org/abs/2002.08909
 
Hochreiter S, Schmidhuber J (1997). Long short-term memory. Neural Comput., 9(8): 1735–1780.
 
Honnibal M, Montani I, Van Landeghem S, Boyd A (2020). spaCy: Industrial-strength Natural Language Processing in Python. Zenodo, https://doi.org/10.5281/zenodo.1212303
 
Koncel-Kedziorski R, Bekal D, Luan Y, Lapata M, Hajishirzi H (2019). Text generation from knowledge graphs with graph transformers. arXiv preprint: https://arxiv.org/abs/1904.02342
 
Lee K, He L, Lewis M, Zettlemoyer L (2017). End-to-end neural coreference resolution. In: Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing (M Palmer, R Hwa, S Riedel, eds.), 188–197. Association for Computational Linguistics, Copenhagen, Denmark.
 
Liu Y, Lapata M (2019). Text summarization with pretrained encoders. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) (K Inui, J Jiang, V Ng, X Wan, eds.), 3730–3740. Association for Computational Linguistics, Hong Kong, China.
 
Logan R, Liu NF, Peters ME, Gardner M, Singh S (2019). Barack’s wife Hillary: Using knowledge graphs for fact-aware language modeling. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (A Korhonen, DR Traum, L Màrquez, eds.), 5962–5971. Association for Computational Linguistics, Florence, Italy.
 
Luan Y, He L, Ostendorf M, Hajishirzi H (2018). Multi-task identification of entities, relations, and coreference for scientific knowledge graph construction. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing (E Riloff, D Chiang, J Hockenmaier, J Tsujii, eds.), 3219–3232. Association for Computational Linguistics, Brussels, Belgium.
 
Manning C, Surdeanu M, Bauer J, Finkel J, Bethard S, McClosky D (2014). The Stanford CoreNLP natural language processing toolkit. In: Proceedings of 52nd Annual Meeting of the Association for Computational Linguistics: System Demonstrations, 55–60. Association for Computational Linguistics, Baltimore, Maryland.
 
Mintz M, Bills S, Snow R, Jurafsky D (2009). Distant supervision for relation extraction without labeled data. In: Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP (J Wiebe, J Su, K-Y Su, eds.), 1003–1011. Association for Computational Linguistics, Suntec, Singapore.
 
Nallapati R, Zhou B, dos Santos C, Gulçehre Ç, Xiang B (2016). Abstractive text summarization using sequence-to-sequence RNNs and beyond. In: Proceedings of The 20th SIGNLL Conference on Computational Natural Language Learning (S Riezler, Y Goldberg, eds.), 280–290. Association for Computational Linguistics, Berlin, Germany.
 
Paulus R, Xiong C, Socher R (2018). A deep reinforced model for abstractive summarization. In: International Conference on Learning Representations.
 
Rush AM, Chopra S, Weston J (2015). A neural attention model for abstractive sentence summarization. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing (L Màrquez, C Callison-Burch, J Su, D Pighin, Y Marton, eds.), 379–389. Association for Computational Linguistics, Lisbon, Portugal.
 
See A, Liu PJ, Manning CD (2017). Get to the point: Summarization with pointer-generator networks. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (R Barzilay, M-Y Kan, eds.), 1073–1083. Association for Computational Linguistics, Vancouver, Canada.
 
Seo M, Kembhavi A, Farhadi A, Hajishirzi H (2017). Bidirectional attention flow for machine comprehension. arXiv preprint: https://arxiv.org/abs/1611.01603
 
Sharma E, Huang L, Hu Z, Wang L (2019). An entity-driven framework for abstractive summarization. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) (K Inui, J Jiang, V Ng, X Wan, eds.), 3280–3291. Association for Computational Linguistics, Hong Kong, China.
 
Speer R, Havasi C (2012). Representing general relational knowledge in ConceptNet 5. In: Proceedings of the Eighth International Conference on Language Resources and Evaluation (LREC’12) (N Calzolari, K Choukri, Declerck T, MU Dogan, B Maegaard, J Mariani, J Odijk, S Piperidis,eds.), 3679–3686. European Language Resources Association (ELRA), Istanbul, Turkey.
 
Tay Y, Bahri D, Metzler D, Juan DC, Zhao Z, Zheng C (2020). Synthesizer: Rethinking self-attention in transformer models. arXiv preprint: https://arxiv.org/abs/2005.00743
 
Trisedya BD, Weikum G, Qi J, Zhang R (2019). Neural relation extraction for knowledge base enrichment. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (A Korhonen, DR Traum, L Màrquez, eds.), 229–240. Association for Computational Linguistics, Florence, Italy.
 
Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, et al. (2017). Attention is all you need. In: Advances in Neural Information Processing Systems (I Guyon, UV Luxburg, S Bengio, H Wallach, R Fergus, S Vishwanathan, R Garnett, eds.), volume 30. Curran Associates, Inc.
 
Wolf T, Debut L, Sanh V, Chaumond J, Delangue C, Moi A, et al. (2019). Huggingface’s transformers: State-of-the-art natural language processing. arXiv preprint: https://arxiv.org/abs/1910.03771
 
Zhang Y, Qi P, Manning CD (2018). Graph convolution over pruned dependency trees improves relation extraction. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing (E Riloff, D Chiang, J Hockenmaier, J Tsujii, eds.), 2205–2215. Association for Computational Linguistics, Brussels, Belgium.

PDF XML
PDF XML

Copyright
© 2021 The Author(s)
This is a free to read article.

Keywords
distant supervise entity extraction graph attention neural network information extraction

Metrics
since February 2021
1207

Article info
views

699

PDF
downloads

Export citation

Copy and paste formatted citation
Placeholder

Download citation in file


Share


RSS

Journal of data science

  • Online ISSN: 1683-8602
  • Print ISSN: 1680-743X

About

  • About journal

For contributors

  • Submit
  • OA Policy
  • Become a Peer-reviewer

Contact us

  • JDS@ruc.edu.cn
  • No. 59 Zhongguancun Street, Haidian District Beijing, 100872, P.R. China
Powered by PubliMill  •  Privacy policy