A Gold Standard Bengali Raw Text Corpus

A Gold Standard Bengali Raw Text Corpus

1 reviews requests (9)
Catalogue Number: 1105
Stock In Stock

OverView

42,37,440 Words | 1,460 Tittles | XML format | 3 domains
Please Login to see the price

Dataset Description

42,37,440 Words | 1,460 Tittles | XML format | 3 domains

Bengali is the official language of West Bengal and Tripura. It belongs to the Indo-Aryan language family. Bengali is influenced by Sanskrit. Greater use of Bengali has contributed to the growth of the language in terms of vocabulary and the number of styles and registers. Bengali is spoken over the whole of West Bengal, Tripura and Bangladesh and in some parts of Bihar, Orissa, and Assam. Bengali refugees, who have settled in Andaman after 1950, have also carried the language there. LDC-IL Bengali Text Corpus developed according to various factors such as quality of the text, representativeness, retrievable format,  size of corpus, authenticity, etc. For collecting text corpus LDC-IL adopts a standard category list of various domains and a prior set of criteria. The corpus of Bengali text can be broadly classified as literary and non- literary texts. A huge amount of literary texts are available in Bengali but scientific texts are less thus LDC-IL attempts to develop balanced text corpora of Bengali. Data has been collected from books, magazines, and newspapers and it is verified true to the original text.

Bengali Text Corpus encoded in a machine-readable form and stored in a standard format. The major encoding being used is Unicode and stored in XML format. The data is embedded with metadata information. The corpus has been created from the contemporary text in a typed method. 


The available Text Corpus details:


Domains

Words

Percentage of

Total Corpus

Aesthetics

40,37,854

95.29 %

Science and Technology 

76,231

1.80 %

Social Sciences

1,23,355 

2.91 %


A detailed explanation of the Bengali Text Corpus will be available in the Bengali Raw Text Corpus Documentation.

For any  research-based citations, please use the following citations:

  • Ramamoorthy, L., Narayan Choudhary, Sonali Sutradhar, Arundhati Sengupta, Sankarshan Dutta, Priyanka Das & Saswati Karmakar. 2019. A Gold Standard Bengali Raw Text Corpus. Central Institute of Indian Languages, Mysore.
  • Choudhary, Narayan & L. Ramamoorthy. 2019. "LDC-IL Raw Text Corpora: An Overview"  in  Linguistic Resources for AI/NLP in Indian Languages, Central Institute of Indian Languages, Mysore. pp. 1-10.

Item specifics

  • Authors Ramamoorthy L., Narayan Choudhary, Sonali Sutradhar, Arundhati Sengupta,Sankarshan Dutta, Priyanka Das, Saswati Karmakar
  • Corpus Type Raw Corpus
  • Catalogue Number 1105
  • ISBN 978-81-7343-204-0
  • Data Source Typed+Cleaned
  • Character Count 25481407
  • Word Count 4237440
  • Release Date 04/04/2019
  • Terms and Conditions General instructions for use of the resources provided by LDC-IL.
Commercial User
Non-Commercial User

Write a review

Please login or register to review