5161927  Words | 739 Titles | XML format | 5 domains.Urdu is one of the prominent language used in the Indian sub-continent. It belongs to the Indo-Aryan family. Urdu is influenced by Arabic and Persian. Urdu is written in the Perso-Arabic script. On the other hand region-wise Urdu language is co-existed side by side mostly in the northern part of India, north-west and eastern parts of India and Pakistan also, although understood and spoken occasionally in the rest of India. Urdu is arisen in the 10th century A.D. due to occupation relations, ethnic exchanges, relocations, and military expeditions. Urdu in India is basically developed in close contact with Persian, which was the language of administration and education during the period of Muslim rule.  LDC-IL Urdu Text Corpus developed according to various factors such as quality of the text, representativeness, retrievable format, size of corpus, authenticity, etc. For collecting text corpus LDC-IL adopts a standard category list of various domains and a prior set of criteria. The corpus of Urdu text can be broadly classified as literary and non- literary texts. A huge amount of literary texts are available in Urdu but scientific texts are less thus LDC-IL attempts to develop balanced text corpora of Urdu. Data has been collected from books, magazines, and newspapers and it is verified true to the original text.The available Text Corpus details: Domains Words Percentage of Total Corpus Aesthetics  2616382 50.69 % Commerce 28601 0.55 % Mass Media 843477 16.34 % Science and Technology 348082 6.74 % Social Sciences 1325385 25.68 % A detailed explanation of the Urdu Text Corpus will be available in the Urdu Text Corpus documentation.For any research-based citations, please use the following citations: Ramamoorthy, L., Narayan Choudhary, Mansoor Khan, Shahnawaz Alam, Bi Bi Mariyam & Rushda Idris Khan. 2019. A Gold Standard Urdu Raw Text Corpus.  Central Institute of Indian Languages, Mysore.Choudhary, Narayan & L. Ramamoorthy. 2019. "LDC-IL Raw Text Corpora: An Overview"  in  Linguistic Resources for AI/NLP in Indian Languages, Central Institute of Indian Languages, Mysore. pp. 1-10...

53,16,552 Words | 499 Tittles | XML format | 5 domainsMaithili is an Indio-Aryan language, a direct descendant of Sanskrit. Which is spoken in the states of Bihar, Jarkhand, and part of Nepal. It is one of the scheduled language of India. LDC-IL Maithili Text Corpus developed according to various factors such as quality of the text, representativeness, retrievable format,  size of corpus, authenticity, etc. For collecting text corpus LDC-IL adopts a standard category list of various domains and a prior set of criteria. The corpus of Maithili text can be broadly classified as literary and non- literary texts. Huge amount of literary texts are available in Maithili but scientific texts are less thus LDC-IL attempts to develop balanced text corpora of Maithili. Data has been collected from books, magazines, and newspapers and it is verified to true to the original texts then warehoused.Maithili Text Corpus encoded in a machine-readable form and stored in a standard format. The major encoding being used is Unicode and stored in XML format. The data is embedded with metadata information. The corpus has been created from the contemporary text in typed and crawled methods.The available Text Corpus details:DomainsWordsPercentage of TotalCorpusAesthetics 38,97,26473.30 %Commerce50,97500.96 %Mass Media12,53,09023.57 %Science and Technology3,13600.06 %Social Sciences1,12,08702.11 %A  detailed explanation of the Maithili Raw Text Corpus will be available in the Maithili Text Corpus Documentation.For any research-based citations, please use the following citations:  Ramamoorthy, L., Narayan  Choudhary, Arun Kumar Singh & Dinesh Mishra. 2019. A Gold Standard Maithili Raw Text Corpus. Central Institute of Indian Languages, Mysore. Choudhary, Narayan & L. Ramamoorthy. 2019. "LDC-IL Raw Text Corpora: An Overview"  in  Linguistic Resources for AI/NLP in Indian Languages, Central Institute of Indian Languages, Mysore. pp. 1-10...

77,63,124 words | 1772 Titles | Data and Metadata in XML format |  6 text domainsKannada is one of the Ancient Indian language which belongs to the Dravidian family. It has its own script. Even though Kannada is considered as a classical language because of its ancient history in literature, the Kannada text corpus is extracted from contemporary text sources. To keep the corpus balanced, the Kannada text corpus is collected by keying-in and proofing text extracts from books of various domains or Crawled from News websites. The available corpus is in Unicode standard and the data with metadata is in XML format. The available Text Corpus details: Domains Words Percentage of Total Corpus Aesthetics  37,78,723 48.68 % Commerce 2,07,053 2.67 % Mass Media 2,07,053 34.54 % Official Document 5,357 0.07 % Science and Technology 2,43,166 3.13 % Social Sciences 8,47,214 10.91 % A detailed explanation of the Kannada Text Corpus will be available in the Kannada Raw Text Corpus Documentation. For any research-based citations, please use the following citations: Ramamoorthy, L., Narayan  Choudhary, Vijayalaxmi F. Patil, Chetan Suryakant Baji,  Malini N Abhyankar, Rajesha N. & Manasa G. 2019. A Gold Standard Kannada Raw Text Corpus. Central Institute of Indian Languages, Mysore. Choudhary, Narayan & L. Ramamoorthy. 2019. "LDC-IL Raw Text Corpora: An Overview"  in  Linguistic Resources for AI/NLP in Indian Languages, Central Institute of Indian Languages, Mysore. pp. 1-10...

1,09,31,902 Words | 1,963 Titles | XML format |  6 text domainsTamil is one of the longest-surviving Classical Languages in the world. It is a Dravidian Language Family. Tamil Text Corpus encoded in a machine-readable form and stored in a standard format. The major encoding being used is Unicode and stored in XML format. The data is embedded with metadata information. The corpus has been created from the contemporary text in typed and crawled methods. Tamil is one of the longest-surviving classical languages in the world. It is a Dravidian language spoken in Tamil Nadu and Sri Lanka, in East-Asian countries like Burma, Malaysia, Singapore, Indonesia, Indio china, Fiji, in South-Africa and British Guinea and in islands like Mauritius and Madagascar etc. The language is an official language in Tamil Nadu and some of the foreign countries such as Sri Lanka and Singapore. It has official status in the Indian state of Tamil Nadu and the Indian Union Territory of Pondicherry. Linguistic Data Consortium for Indian Languages (LDC-IL) Tamil Text Corpus developed according to various factors such as quality of the text, representativeness, retrievable format,  size of corpus, authenticity, etc. For collecting text corpus LDC-IL adopts a standard category list of various domains and a prior set of criteria. The corpus of Tamil text can be broadly classified as literary and non- literary texts. A huge amount of literary texts are available in Tamil but scientific texts are less, thus LDC-IL attempts to develop balanced text corpora of Tamil. Data has been collected from books, Magazines, and Newspapers and it is verified to true to the original texts then warehoused.The available Text Corpus details are as follows: Domains Words Percentage of Total Corpus Aesthetics  55,95,316 51.18 % Commerce 83,148 00.76 % Mass Media 21,00,226 19.21 % Official Document 12,768 0.12 % Science and Technology 88,65,532 8.11 % Social Sciences 22,53,912 20.62 % A  detailed explanation of the Tamil Raw Text Corpus will be available in the Tamil Text Corpus Documentation.For any research-based citations, please use the following citations: Ramamoorthy, L., Narayan Choudhary, G. Palanirajan, S. Thennarasu, Prem Kumar L. R, Amudha R., Prabagaran R., Vijayan N. & M. Ramesh Kumar. 2019. A Gold Standard Tamil Raw Text Corpus.  Central Institute of Indian Languages, Mysore.Choudhary, Narayan & L. Ramamoorthy. 2019. "LDC-IL Raw Text Corpora: An Overview"  in  Linguistic Resources for AI/NLP in Indian Languages, Central Institute of Indian Languages, Mysore. pp. 1-10...