Automated Reading Comprehension System Project

• Tentative Plan/Steps
  • Try to reproduce Yao et al model
  • QAG Model
    • First identifies potential answers excerpted from passage
    • Inputs into some BERT model
      • Reverse engineers a question to fit to the pair
    • Filters out QA pairs which don’t make sense
• BERT big picture
  • Encoder
    • Input is the Story + Answer
  • Decoder
    • Accepts a start and runs through encoder
    • Repeats process until end?
• Resources

  • Shows how transformers generates a sequence to sequence output

    Google Colaboratory

  • https://github.com/WorkInTheDark/FairytaleQA_Dataset

  • https://github.com/WorkInTheDark/FairytaleQA_QAG_System

    • Should provide a completed model to use as well

  • Sequence to Sequence introduction

    https://proceedings.neurips.cc/paper/2014/file/a14ac55a4f27472c5d894ec1c3c743d2-Paper.pdf

    • Includes mathematical understanding

  • Example code on how to train transformers

    • https://github.com/huggingface/transformers/tree/main/examples/pytorch
    • Shows how to train for specific tasks
      • Example summarization
      • Example summarization