ArXiv Miner

What is ArXiv-Miner
arxiv-miner is a quick handy library that helps power Sci-Genie. Sci-Genie is a search engine for quickly searching through full text of papers on CS ArXiv. arxiv-miner helps extract and parse LaTeX documents from CS ArXiv. It also supports storage and search of those parsed documents using Elasticsearch. The library can be applicable for all other domains like Math, Physics, Biology etc.

Why was ArXiv-Miner created ?
ArXiv Miner was created for easily scraping, parsing and searching research content on ArXiv. This library was created after stitching together a solutions from the code of various tools like arxiv-sanity, arxiv-vanity/engrafo, arxivscraper, tex2py, cso-classifier and axcell. Parsed structure of the content can be useful in search or any scientific research mining/AI applications as a heuristic baseline.
— Lees op arxiv-miner.turing-bot.com/