Hindi is a Major, Indo-Aryan language, a descendent of Sanskrit, which is spoken in the central and northern India.
Hindi Text Corpus encoded in a machine readable form and stored in a standard format. The major encoding being used is Unicode and stored in XML format.The data is embedded with metadatainformation. The corpus has been created from contemporary text in typed and crawled methods.LDC-IL Hindi Text Corpus size is 10317177 words drawn from 1223 different titles. The four major domains are Aesthetics, Social Sciences, Mass Media and Science & Technology.
Hindi is a Major, Indo-Aryan language, a descendent of Sanskrit, which is spoken in the central and northern India, in the states of Bihar, Chhattisgarh, Delhi, Haryana, Himachal Pradesh, Jharkhand, Madhya Pradesh, Rajasthan, Uttarakhand and Uttar Pradesh. LDC-IL Hindi Text Corpus developed according to various factors such as quality of the text, representativeness, retrievable format, size of corpus, authenticity etc. For collecting text corpus LDC-IL adopts a standard category list of various domains and a prior set of criteria. The corpus of Hindi text can be broadly classified as literary and non- literary texts. Huge amount of literary texts are available in Hindi but scientific texts are less thus LDC-IL attempts to develop balanced text corpora of Hindi. Data has been collected from books, magazines and newspapers and it is verified to true to the original texts then warehoused.
A much more detailed explanation of the Hindi Text Corpus will be available in the Hindi Text Data Documentation.
For any research based citations, please use the following citations:
- Ramamoorthy, L., Narayan Choudhary, Jitendra Kumar Singh, Richa, Anjali Sinha, Dheeraj Kumar Mishra, Arimardan Kumar Tripathi, Aditi Debsharma, Satyaendra Kumar Awasthi & Madhupriya Pathak. 2019. A Gold Standard Hindi Raw Text Corpus. Central Institute of Indian Languages, Mysore.
- Choudhary, Narayan & L. Ramamoorthy. 2019. "LDC-IL Raw Text Corpora: An Overview" in Linguistic Resources for AI/NLP in Indian Languages, Central Institute of Indian Languages, Mysore. pp. 1-10.
- Authors Ramamoorthy L., Narayan Choudhary, Satyaendra Awasthi, Madhu Priya Pathak, Jitendra Kumar Singh, Richa, Anjali Sinha, Dheeraj Kumar Mishra,Arimardan Kumar Tripathi, Sumedha Shukla
- Corpus Type Raw Corpus
- Catalogue Number 1120
- ISBN 978-81-7343-219-4
- Data Source Typed+Cleaned
- Character Count 52569629
- Word Count 10317177
- Release Date 04/04/2019
- Terms and Conditions General instructions for use of the resources provided by LDC-IL.