Hi !
The url works on my side.

Is the url working in your navigator ?
Are you connected to internet ? Does your network block access to `raw.githubusercontent.com` ?

> Hi !
> The url works on my side.
>
> Is the url working in your navigator ?
> Are you connected to internet ? Does your network block access to `raw.githubusercontent.com` ?

I tried another server, it's working now. Thanks a lot.

And I'm curious about why download things from "github" when I load dataset from local files ?  Dose datasets work if my network crashed?

It seems my network frequently crashed so most time it cannot work.

> > Hi !
> > The url works on my side.
> > Is the url working in your navigator ?
> > Are you connected to internet ? Does your network block access to `raw.githubusercontent.com` ?
>
> I tried another server, it's working now. Thanks a lot.
>
> And I'm curious about why download things from "github" when I load dataset from local files ? Dose datasets work if my network crashed?

I download the scripts `https://raw.githubusercontent.com/huggingface/datasets/1.1.2/datasets/csv/csv.py` and move it to the package dir `*/datasets/` solved the problem. Could you please put the file `datasets/datasets/csv/csv.py` to `datasets/src/datasets/`？

Thanks :D

hello, how did you solve this problems?

> > > Hi !
> > > The url works on my side.
> > > Is the url working in your navigator ?
> > > Are you connected to internet ? Does your network block access to `raw.githubusercontent.com` ?
> >
> >
> > I tried another server, it's working now. Thanks a lot.
> > And I'm curious about why download things from "github" when I load dataset from local files ? Dose datasets work if my network crashed?
>
> I download the scripts `https://raw.githubusercontent.com/huggingface/datasets/1.1.2/datasets/csv/csv.py` and move it to the package dir `*/datasets/` solved the problem. Could you please put the file `datasets/datasets/csv/csv.py` to `datasets/src/datasets/`？
>
> Thanks :D

hello, I tried this. but it still failed. how do you fix this error?

> hello, how did you solve this problems?
>
> > > > Hi !
> > > > The url works on my side.
> > > > Is the url working in your navigator ?
> > > > Are you connected to internet ? Does your network block access to `raw.githubusercontent.com` ?
> > >
> > >
> > > I tried another server, it's working now. Thanks a lot.
> > > And I'm curious about why download things from "github" when I load dataset from local files ? Dose datasets work if my network crashed?
> >
> >
> > I download the scripts `https://raw.githubusercontent.com/huggingface/datasets/1.1.2/datasets/csv/csv.py` and move it to the package dir `*/datasets/` solved the problem. Could you please put the file `datasets/datasets/csv/csv.py` to `datasets/src/datasets/`？
> > Thanks :D
>
> hello, I tried this. but it still failed. how do you fix this error?

你把那个脚本下载到你本地安装目录下，然后 `load_dataset(csv_script_path, data_fiels)`

> > hello, how did you solve this problems?
> > > > > Hi !
> > > > > The url works on my side.
> > > > > Is the url working in your navigator ?
> > > > > Are you connected to internet ? Does your network block access to `raw.githubusercontent.com` ?
> > > >
> > > >
> > > > I tried another server, it's working now. Thanks a lot.
> > > > And I'm curious about why download things from "github" when I load dataset from local files ? Dose datasets work if my network crashed?
> > >
> > >
> > > I download the scripts `https://raw.githubusercontent.com/huggingface/datasets/1.1.2/datasets/csv/csv.py` and move it to the package dir `*/datasets/` solved the problem. Could you please put the file `datasets/datasets/csv/csv.py` to `datasets/src/datasets/`？
> > > Thanks :D
> >
> >
> > hello, I tried this. but it still failed. how do you fix this error?
>
> 你把那个脚本下载到你本地安装目录下，然后 `load_dataset(csv_script_path, data_fiels)`

好的好的！解决了，感谢感谢！！！

>
>
> > hello, how did you solve this problems?
> > > > > Hi !
> > > > > The url works on my side.
> > > > > Is the url working in your navigator ?
> > > > > Are you connected to internet ? Does your network block access to `raw.githubusercontent.com` ?
> > > >
> > > >
> > > > I tried another server, it's working now. Thanks a lot.
> > > > And I'm curious about why download things from "github" when I load dataset from local files ? Dose datasets work if my network crashed?
> > >
> > >
> > > I download the scripts `https://raw.githubusercontent.com/huggingface/datasets/1.1.2/datasets/csv/csv.py` and move it to the package dir `*/datasets/` solved the problem. Could you please put the file `datasets/datasets/csv/csv.py` to `datasets/src/datasets/`？
> > > Thanks :D
> >
> >
> > hello, I tried this. but it still failed. how do you fix this error?
>
> 你把那个脚本下载到你本地安装目录下，然后 `load_dataset(csv_script_path, data_fiels)`

我照着做了，然后报错。
ValueError: unable to parse C:/Software/Anaconda/envs/ptk_gpu2/Lib/site-packages/datasets\dataset_infos.json as a URL or as a local path

`---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-5-fd2106a3f053> in <module>
----> 1 dataset = load_dataset('C:/Software/Anaconda/envs/ptk_gpu2/Lib/site-packages/datasets/csv.py', data_files='./test.csv', delimiter=',', autogenerate_column_names=False)

C:\Software\Anaconda\envs\ptk_gpu2\lib\site-packages\datasets\load.py in load_dataset(path, name, data_dir, data_files, split, cache_dir, features, download_config, download_mode, ignore_verifications, save_infos, script_version, **config_kwargs)
    588     # Download/copy dataset processing script
    589     module_path, hash = prepare_module(
--> 590         path, script_version=script_version, download_config=download_config, download_mode=download_mode, dataset=True
    591     )
    592

C:\Software\Anaconda\envs\ptk_gpu2\lib\site-packages\datasets\load.py in prepare_module(path, script_version, download_config, download_mode, dataset, force_local_path, **download_kwargs)
    296         local_dataset_infos_path = cached_path(
    297             dataset_infos,
--> 298             download_config=download_config,
    299         )
    300     except (FileNotFoundError, ConnectionError):

C:\Software\Anaconda\envs\ptk_gpu2\lib\site-packages\datasets\utils\file_utils.py in cached_path(url_or_filename, download_config, **download_kwargs)
    316     else:
    317         # Something unknown
--> 318         raise ValueError("unable to parse {} as a URL or as a local path".format(url_or_filename))
    319
    320     if download_config.extract_compressed_file and output_path is not None:

ValueError: unable to parse C:/Software/Anaconda/envs/ptk_gpu2/Lib/site-packages/datasets\dataset_infos.json as a URL or as a local path

`

I also experienced this issue this morning. Looks like something specific to windows.
I'm working on a fix

>
>
> I opened a PR @wn1652400018

Thanks you!, It works very well.

## load_dataset for LOCAL CSV files report CONNECTION ERROR
- **Description:**
A local demo csv file:
```
import pandas as pd
import numpy as np
from datasets import load_dataset
import torch
import transformers

df = pd.DataFrame(np.arange(1200).reshape(300,4))
df.to_csv('test.csv', header=False, index=False)

print('datasets version: ', datasets.__version__)
print('pytorch version: ', torch.__version__)
print('transformers version: ', transformers.__version__)

# output:
datasets version:  1.1.2
pytorch version:  1.5.0
transformers version:  3.2.0
```

when I load data through `dataset`:
```
dataset = load_dataset('csv', data_files='./test.csv', delimiter=',', autogenerate_column_names=False)
```
Error infos:
```
ConnectionError                           Traceback (most recent call last)
<ipython-input-17-bbdadb9a0c78> in <module>
----> 1 dataset = load_dataset('csv', data_files='./test.csv', delimiter=',', autogenerate_column_names=False)

~/.conda/envs/py36/lib/python3.6/site-packages/datasets/load.py in load_dataset(path, name, data_dir, data_files, split, cache_dir, features, download_config, download_mode, ignore_verifications, save_infos, script_version, **config_kwargs)
    588     # Download/copy dataset processing script
    589     module_path, hash = prepare_module(
--> 590         path, script_version=script_version, download_config=download_config, download_mode=download_mode, dataset=True
    591     )
    592

~/.conda/envs/py36/lib/python3.6/site-packages/datasets/load.py in prepare_module(path, script_version, download_config, download_mode, dataset, force_local_path, **download_kwargs)
    266         file_path = hf_github_url(path=path, name=name, dataset=dataset, version=script_version)
    267         try:
--> 268             local_path = cached_path(file_path, download_config=download_config)
    269         except FileNotFoundError:
    270             if script_version is not None:

~/.conda/envs/py36/lib/python3.6/site-packages/datasets/utils/file_utils.py in cached_path(url_or_filename, download_config, **download_kwargs)
    306             user_agent=download_config.user_agent,
    307             local_files_only=download_config.local_files_only,
--> 308             use_etag=download_config.use_etag,
    309         )
    310     elif os.path.exists(url_or_filename):

~/.conda/envs/py36/lib/python3.6/site-packages/datasets/utils/file_utils.py in get_from_cache(url, cache_dir, force_download, proxies, etag_timeout, resume_download, user_agent, local_files_only, use_etag)
    473         elif response is not None and response.status_code == 404:
    474             raise FileNotFoundError("Couldn't find file at {}".format(url))
--> 475         raise ConnectionError("Couldn't reach {}".format(url))
    476
    477     # Try a second time

ConnectionError: Couldn't reach https://raw.githubusercontent.com/huggingface/datasets/1.1.2/datasets/csv/csv.py
```

And I try to connect to the site with requests:
```
import requests

requests.head("https://raw.githubusercontent.com/huggingface/datasets/1.1.2/datasets/csv/csv.py")
```

Similarly Error occurs:
```
---------------------------------------------------------------------------
ConnectionRefusedError                    Traceback (most recent call last)
~/.conda/envs/py36/lib/python3.6/site-packages/urllib3/connection.py in _new_conn(self)
    159             conn = connection.create_connection(
--> 160                 (self._dns_host, self.port), self.timeout, **extra_kw
    161             )

~/.conda/envs/py36/lib/python3.6/site-packages/urllib3/util/connection.py in create_connection(address, timeout, source_address, socket_options)
     83     if err is not None:
---> 84         raise err
     85

~/.conda/envs/py36/lib/python3.6/site-packages/urllib3/util/connection.py in create_connection(address, timeout, source_address, socket_options)
     73                 sock.bind(source_address)
---> 74             sock.connect(sa)
     75             return sock

ConnectionRefusedError: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

NewConnectionError                        Traceback (most recent call last)
~/.conda/envs/py36/lib/python3.6/site-packages/urllib3/connectionpool.py in urlopen(self, method, url, body, headers, retries, redirect, assert_same_host, timeout, pool_timeout, release_conn, chunked, body_pos, **response_kw)
    676                 headers=headers,
--> 677                 chunked=chunked,
    678             )

~/.conda/envs/py36/lib/python3.6/site-packages/urllib3/connectionpool.py in _make_request(self, conn, method, url, timeout, chunked, **httplib_request_kw)
    380         try:
--> 381             self._validate_conn(conn)
    382         except (SocketTimeout, BaseSSLError) as e:

~/.conda/envs/py36/lib/python3.6/site-packages/urllib3/connectionpool.py in _validate_conn(self, conn)
    975         if not getattr(conn, "sock", None):  # AppEngine might not have  `.sock`
--> 976             conn.connect()
    977

~/.conda/envs/py36/lib/python3.6/site-packages/urllib3/connection.py in connect(self)
    307         # Add certificate verification
--> 308         conn = self._new_conn()
    309         hostname = self.host

~/.conda/envs/py36/lib/python3.6/site-packages/urllib3/connection.py in _new_conn(self)
    171             raise NewConnectionError(
--> 172                 self, "Failed to establish a new connection: %s" % e
    173             )

NewConnectionError: <urllib3.connection.HTTPSConnection object at 0x7f3cceda5e48>: Failed to establish a new connection: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

MaxRetryError                             Traceback (most recent call last)
~/.conda/envs/py36/lib/python3.6/site-packages/requests/adapters.py in send(self, request, stream, timeout, verify, cert, proxies)
    448                     retries=self.max_retries,
--> 449                     timeout=timeout
    450                 )

~/.conda/envs/py36/lib/python3.6/site-packages/urllib3/connectionpool.py in urlopen(self, method, url, body, headers, retries, redirect, assert_same_host, timeout, pool_timeout, release_conn, chunked, body_pos, **response_kw)
    724             retries = retries.increment(
--> 725                 method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
    726             )

~/.conda/envs/py36/lib/python3.6/site-packages/urllib3/util/retry.py in increment(self, method, url, response, error, _pool, _stacktrace)
    438         if new_retry.is_exhausted():
--> 439             raise MaxRetryError(_pool, url, error or ResponseError(cause))
    440

MaxRetryError: HTTPSConnectionPool(host='raw.githubusercontent.com', port=443): Max retries exceeded with url: /huggingface/datasets/1.1.2/datasets/csv/csv.py (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f3cceda5e48>: Failed to establish a new connection: [Errno 111] Connection refused',))

During handling of the above exception, another exception occurred:

ConnectionError                           Traceback (most recent call last)
<ipython-input-20-18cc3eb4a049> in <module>
      1 import requests
      2
----> 3 requests.head("https://raw.githubusercontent.com/huggingface/datasets/1.1.2/datasets/csv/csv.py")

~/.conda/envs/py36/lib/python3.6/site-packages/requests/api.py in head(url, **kwargs)
    102
    103     kwargs.setdefault('allow_redirects', False)
--> 104     return request('head', url, **kwargs)
    105
    106

~/.conda/envs/py36/lib/python3.6/site-packages/requests/api.py in request(method, url, **kwargs)
     59     # cases, and look like a memory leak in others.
     60     with sessions.Session() as session:
---> 61         return session.request(method=method, url=url, **kwargs)
     62
     63

~/.conda/envs/py36/lib/python3.6/site-packages/requests/sessions.py in request(self, method, url, params, data, headers, cookies, files, auth, timeout, allow_redirects, proxies, hooks, stream, verify, cert, json)
    528         }
    529         send_kwargs.update(settings)
--> 530         resp = self.send(prep, **send_kwargs)
    531
    532         return resp

~/.conda/envs/py36/lib/python3.6/site-packages/requests/sessions.py in send(self, request, **kwargs)
    641
    642         # Send the request
--> 643         r = adapter.send(request, **kwargs)
    644
    645         # Total elapsed time of the request (approximately)

~/.conda/envs/py36/lib/python3.6/site-packages/requests/adapters.py in send(self, request, stream, timeout, verify, cert, proxies)
    514                 raise SSLError(e, request=request)
    515
--> 516             raise ConnectionError(e, request=request)
    517
    518         except ClosedPoolError as e:

ConnectionError: HTTPSConnectionPool(host='raw.githubusercontent.com', port=443): Max retries exceeded with url: /huggingface/datasets/1.1.2/datasets/csv/csv.py (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f3cceda5e48>: Failed to establish a new connection: [Errno 111] Connection refused',))
```


Quail dataset urls are out of date

Hi ! Thanks for reporting.
We should fix the urls and use quail 1.3.
If you want to contribute feel free to fix the urls and open a PR :)

Done! PR [https://github.com/huggingface/datasets/pull/820](https://github.com/huggingface/datasets/pull/820)

Updated links and also regenerated the metadata and dummy data for v1.3 in order to pass verifications as described here: [https://huggingface.co/docs/datasets/share_dataset.html#adding-tests-and-metadata-to-the-dataset](https://huggingface.co/docs/datasets/share_dataset.html#adding-tests-and-metadata-to-the-dataset).

Closing since #820 is merged.
Thanks again for fixing the urls :)

Code

```
from datasets import load_dataset
quail = load_dataset('quail')
```

Error

```
FileNotFoundError: Couldn't find file at https://raw.githubusercontent.com/text-machine-lab/quail/master/quail_v1.2/xml/ordered/quail_1.2_train.xml
```

As per [quail v1.3 commit](https://github.com/text-machine-lab/quail/commit/506501cfa34d9ec6c042d31026ba6fea6bcec8ff) it looks like the location and suggested ordering has changed. In [https://github.com/huggingface/datasets/blob/master/datasets/quail/quail.py#L52-L58](https://github.com/huggingface/datasets/blob/master/datasets/quail/quail.py#L52-L58) the quail v1.2 datasets are being pointed to, which don't exist anymore.


On loading a metric from datasets, I get the following error

Hi ! We support only pyarrow > 0.17.1 so that we have access to the `PyExtensionType` object.
Could you update pyarrow and try again ?
```
pip install --upgrade pyarrow
```

`from datasets import load_metric`

`metric = load_metric('bleurt')`

Traceback:
210 class _ArrayXDExtensionType(pa.PyExtensionType):
    211
    212     ndims: int = None

AttributeError: module 'pyarrow' has no attribute 'PyExtensionType'

Any help will be appreciated. Thank you.


Empty output/answer in TriviaQA test set (both in 'kilt_tasks' and 'trivia_qa')

Yes: TriviaQA has a private test set for the leaderboard [here](https://competitions.codalab.org/competitions/17208)

For the KILT training and validation portions, you need to link the examples from the TriviaQA dataset as detailed here:
https://github.com/huggingface/datasets/blob/master/datasets/kilt_tasks/README.md

Oh ok, I guess I read the paper too fast 😅, thank you for your answer!

# The issue

It's all in the title, it appears to be fine on the train and validation sets.

Is there some kind of mapping to do like for the questions (see https://github.com/huggingface/datasets/blob/master/datasets/kilt_tasks/README.md) ?

# How to reproduce
```py
from datasets import load_dataset
kilt_tasks = load_dataset("kilt_tasks")
trivia_qa = load_dataset('trivia_qa', 'unfiltered.nocontext')
# both in "kilt_tasks"
In [18]: any([output['answer'] for output in kilt_tasks['test_triviaqa']['output']])
Out[18]: False
# and "trivia_qa"
In [13]: all([answer['value'] == '<unk>' for answer in trivia_qa['test']['answer']])
Out[13]: True
# appears to be fine on the train and validation sets.
In [14]: all([answer['value'] == '<unk>' for answer in trivia_qa['train']['answer']])
Out[14]: False

In [15]: all([answer['value'] == '<unk>' for answer in trivia_qa['validation']['answer']])
Out[15]: False

In [16]: any([output['answer'] for output in kilt_tasks['train_triviaqa']['output']])
Out[16]: True

In [17]: any([output['answer'] for output in kilt_tasks['validation_triviaqa']['output']])
Out[17]: True

```


fix: typos in tutorial to map KILT and TriviaQA


Add XGlue

Really cool to add XGlue, this will be a nice addition !

Splits shouldn't depend on the language. There must be configurations for each language, as we're doing for xnli, xtreme, etc.
For example for XGlue we'll have these configurations: NER.de, NER.en etc.

Dataset is ready to merge. An important feature of this dataset is that for each config the train data is in English, while dev and test data are in multiple languages. Therefore, @lhoestq and I decided offline that we will give the dataset the following API, *e.g.* for

```python
load_dataset("xglue", "ner")  # would give the splits 'train', 'validation.en', 'test.en', 'validation.es', 'test.es', ...
```
=> therefore one can load a single language test via

```python
load_dataset("xglue", "ner", split="test.es")
```

Close #749.


How to join two datasets?

Hi ! Currently the only way to add new fields to a dataset is by using `.map` and picking items from the other dataset

Closing this one. Feel free to re-open if you have other questions about this issue.

Also linking another discussion about joining datasets: #853

Hi,

I'm wondering if it's possible to join two (preprocessed) datasets with the same number of rows but different labels?

I'm currently trying to create paired sentences for BERT from `wikipedia/'20200501.en`, and I couldn't figure out a way to create a paired sentence using `.map()` where the second sentence is **not** the next sentence (i.e., from a different article) of the first sentence.

Thanks!


Update loading_metrics.rst

Minor bug


switch amazon reviews class label order

Switches the label order to be more intuitive for amazon reviews, #791.


Cannot load TREC dataset: ConnectionError

dataset bug

A bug in a dataset script provided in the library

Hi ! Indeed there's an issue with those links.
We should probably use the target urls of the redirections instead

Hi, the same issue here, could you tell me how to download it through datasets? thanks

Actually it's already fixed on the master branch since #740
I'll do the 1.1.3 release soon

Hi
thanks, but I did tried to install from the pip install git+... and it does
not work for me,. thanks for the help. I have the same issue with wmt16,
"ro-en"
thanks.
Best
Rabeeh

On Mon, Nov 16, 2020 at 10:29 AM Quentin Lhoest <notifications@github.com>
wrote:

> Actually it's already fixed on the master branch since #740
> <https://github.com/huggingface/datasets/pull/740>
> I'll do the 1.1.3 release soon
>
> —
> You are receiving this because you commented.
> Reply to this email directly, view it on GitHub
> <https://github.com/huggingface/datasets/issues/798#issuecomment-727854736>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/ABP4ZCEUBJKPOCLABXCKMPDSQDWH3ANCNFSM4TJBUKSA>
> .
>

I just tested on google colab using
```python
!pip install git+https://github.com/huggingface/datasets.git
from datasets import load_dataset
load_dataset("trec")
```
and it works.
Can you detail how you got the issue even when using the latest version on master ?

Also about wmt we'll look into it, thanks for reporting !

I think the new url with .edu is also broken:
```
ConnectionError: Couldn't reach https://cogcomp.seas.upenn.edu/Data/QA/QC/train_5500.label
```
Cant download the dataset anymore.

Hi ! The URL seems to work fine on my side, can you try again ?

Forgot to update, i wrote an email to the webmaster of seas.upenn.edu because i couldnt reach the url on any machine. This was the answer:
```
Thank you for your report. The server was offline for maintenance and is now available again.
```
Guess all back to normal now 🙂

## Problem
I cannot load "trec" dataset, it results with ConnectionError as shown below. I've tried on both Google Colab and locally.
* `requests.head('http://cogcomp.org/Data/QA/QC/train_5500.label')` returns <Response [302]>.
* `requests.head('http://cogcomp.org/Data/QA/QC/train_5500.label', allow_redirects=True)` raises `requests.exceptions.TooManyRedirects: Exceeded 30 redirects.`
* Opening `http://cogcomp.org/Data/QA/QC/train_5500.label' in a browser works, but opens a different address
* Increasing max_redirects to 100 doesn't help

Also, while debugging I've seen that requesting 'https://storage.googleapis.com/huggingface-nlp/cache/datasets/trec/default/1.1.0/dataset_info.json' returns <Response [404]> before, but it doesn't raise any errors. Not sure if that's relevant.

* datasets.__version__ == '1.1.2'
* requests.__version__ == '2.24.0'

## Error trace
```
>>> import datasets
>>> datasets.__version__
'1.1.2'
>>> dataset = load_dataset("trec", split="train")
Using custom data configuration default
Downloading and preparing dataset trec/default (download: 350.79 KiB, generated: 403.39 KiB, post-processed: Unknown size, total: 754.18 KiB) to /home/przemyslaw/.cache/huggingface/datasets/trec/default/1.1.0/ca4248481ad244f235f4cf277186cad2ee8769f975119a2bbfc41b8932b88bd7...
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/przemyslaw/.local/lib/python3.6/site-packages/datasets/load.py", line 611, in load_dataset
    ignore_verifications=ignore_verifications,
  File "/home/przemyslaw/.local/lib/python3.6/site-packages/datasets/builder.py", line 476, in download_and_prepare
    dl_manager=dl_manager, verify_infos=verify_infos, **download_and_prepare_kwargs
  File "/home/przemyslaw/.local/lib/python3.6/site-packages/datasets/builder.py", line 531, in _download_and_prepare
    split_generators = self._split_generators(dl_manager, **split_generators_kwargs)
  File "/home/przemyslaw/.cache/huggingface/modules/datasets_modules/datasets/trec/ca4248481ad244f235f4cf277186cad2ee8769f975119a2bbfc41b8932b88bd7/trec.py", line 140, in _split_generators
    dl_files = dl_manager.download_and_extract(_URLs)
  File "/home/przemyslaw/.local/lib/python3.6/site-packages/datasets/utils/download_manager.py", line 254, in download_and_extract
    return self.extract(self.download(url_or_urls))
  File "/home/przemyslaw/.local/lib/python3.6/site-packages/datasets/utils/download_manager.py", line 179, in download
    num_proc=download_config.num_proc,
  File "/home/przemyslaw/.local/lib/python3.6/site-packages/datasets/utils/py_utils.py", line 225, in map_nested
    _single_map_nested((function, obj, types, None, True)) for obj in tqdm(iterable, disable=disable_tqdm)
  File "/home/przemyslaw/.local/lib/python3.6/site-packages/datasets/utils/py_utils.py", line 225, in <listcomp>
    _single_map_nested((function, obj, types, None, True)) for obj in tqdm(iterable, disable=disable_tqdm)
  File "/home/przemyslaw/.local/lib/python3.6/site-packages/datasets/utils/py_utils.py", line 163, in _single_map_nested
    return function(data_struct)
  File "/home/przemyslaw/.local/lib/python3.6/site-packages/datasets/utils/file_utils.py", line 308, in cached_path
    use_etag=download_config.use_etag,
  File "/home/przemyslaw/.local/lib/python3.6/site-packages/datasets/utils/file_utils.py", line 475, in get_from_cache
    raise ConnectionError("Couldn't reach {}".format(url))
ConnectionError: Couldn't reach http://cogcomp.org/Data/QA/QC/train_5500.label
```

I would appreciate some suggestions here.


Token classification labels are strings and we don't have the list of labels

enhancement

New feature or request

Dataset discussion

Discussions on the datasets

Indeed. Pinging @stefan-it here if he want to give an expert opinion :)

Not sure if this is an issue we want to fix or not, putting it here so it's not forgotten. Right now, in token classification datasets, the labels for NER, POS and the likes are typed as `Sequence` of `strings`, which is wrong in my opinion. These should be `Sequence` of `ClassLabel` or some types that gives easy access to the underlying labels.

The main problem for preprocessing those datasets is that the list of possible labels is not stored inside the `Dataset` object which makes converting the labels to IDs quite difficult (you either have to know the list of labels in advance or run a full pass through the dataset to get the list of labels, the `unique` method being useless with the type `Sequence[str]`).


Descriptions of raw and processed versions of wikitext are inverted

Nothing of importance, but it looks like the descriptions of wikitext-n-v1 and wikitext-n-raw-v1 are inverted for both n=2 and n=103. I just verified by loading them and the `<unk>` tokens are present in the non-raw versions, which confirms that it's a mere inversion of the descriptions and not of the datasets themselves.

Also it would be nice if those descriptions appeared in the dataset explorer.

https://github.com/huggingface/datasets/blob/87bd0864845ea0a1dd7167918dc5f341bf807bd3/datasets/wikitext/wikitext.py#L52


bug

Something isn't working

Hi ! Thanks for reporting that's a bug on master indeed.
We'll fix that soon

Hi,

Currently I am trying to load csv file with customized read_options. And the latest master seems broken if we pass the ReadOptions object.

Here is a code snippet
```python
from datasets import load_dataset
from pyarrow.csv import ReadOptions
load_dataset("csv", data_files=["out.csv"], read_options=ReadOptions(block_size=16*1024*1024))
```
error is `self.options cannot be converted to a Python object for pickling`
Would you mind to take a look? Thanks!

```
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-28-ab83fec2ded4> in <module>
----> 1 load_dataset("csv", data_files=["out.csv"], read_options=ReadOptions(block_size=16*1024*1024))

/tmp/datasets/src/datasets/load.py in load_dataset(path, name, data_dir, data_files, split, cache_dir, features, download_config, download_mode, ignore_verifications, save_infos, script_version, **config_kwargs)
    602         hash=hash,
    603         features=features,
--> 604         **config_kwargs,
    605     )
    606

/tmp/datasets/src/datasets/builder.py in __init__(self, cache_dir, name, hash, features, **config_kwargs)
    162             name,
    163             custom_features=features,
--> 164             **config_kwargs,
    165         )
    166

/tmp/datasets/src/datasets/builder.py in _create_builder_config(self, name, custom_features, **config_kwargs)
    281                 )
    282             else:
--> 283                 suffix = Hasher.hash(config_kwargs_to_add_to_suffix)
    284
    285         if builder_config.data_files is not None:

/tmp/datasets/src/datasets/fingerprint.py in hash(cls, value)
     51             return cls.dispatch[type(value)](cls, value)
     52         else:
---> 53             return cls.hash_default(value)
     54
     55     def update(self, value):

/tmp/datasets/src/datasets/fingerprint.py in hash_default(cls, value)
     44     @classmethod
     45     def hash_default(cls, value):
---> 46         return cls.hash_bytes(dumps(value))
     47
     48     @classmethod

/tmp/datasets/src/datasets/utils/py_utils.py in dumps(obj)
    365     file = StringIO()
    366     with _no_cache_fields(obj):
--> 367         dump(obj, file)
    368     return file.getvalue()
    369

/tmp/datasets/src/datasets/utils/py_utils.py in dump(obj, file)
    337 def dump(obj, file):
    338     """pickle an object to a file"""
--> 339     Pickler(file, recurse=True).dump(obj)
    340     return
    341

~/.local/lib/python3.6/site-packages/dill/_dill.py in dump(self, obj)
    444             raise PicklingError(msg)
    445         else:
--> 446             StockPickler.dump(self, obj)
    447         stack.clear()  # clear record of 'recursion-sensitive' pickled objects
    448         return

/usr/lib/python3.6/pickle.py in dump(self, obj)
    407         if self.proto >= 4:
    408             self.framer.start_framing()
--> 409         self.save(obj)
    410         self.write(STOP)
    411         self.framer.end_framing()

/usr/lib/python3.6/pickle.py in save(self, obj, save_persistent_id)
    474         f = self.dispatch.get(t)
    475         if f is not None:
--> 476             f(self, obj) # Call unbound method with explicit self
    477             return
    478

~/.local/lib/python3.6/site-packages/dill/_dill.py in save_module_dict(pickler, obj)
    931             # we only care about session the first pass thru
    932             pickler._session = False
--> 933         StockPickler.save_dict(pickler, obj)
    934         log.info("# D2")
    935     return

/usr/lib/python3.6/pickle.py in save_dict(self, obj)
    819
    820         self.memoize(obj)
--> 821         self._batch_setitems(obj.items())
    822
    823     dispatch[dict] = save_dict

/usr/lib/python3.6/pickle.py in _batch_setitems(self, items)
    850                 k, v = tmp[0]
    851                 save(k)
--> 852                 save(v)
    853                 write(SETITEM)
    854             # else tmp is empty, and we're done

/usr/lib/python3.6/pickle.py in save(self, obj, save_persistent_id)
    494             reduce = getattr(obj, "__reduce_ex__", None)
    495             if reduce is not None:
--> 496                 rv = reduce(self.proto)
    497             else:
    498                 reduce = getattr(obj, "__reduce__", None)

~/.local/lib/python3.6/site-packages/pyarrow/_csv.cpython-36m-x86_64-linux-gnu.so in pyarrow._csv.ReadOptions.__reduce_cython__()

TypeError: self.options cannot be converted to a Python object for pickling
```


[Datasets] fix discofuse links

The discofuse links were changed: https://github.com/google-research-datasets/discofuse/commit/d27641016eb5b3eb2af03c7415cfbb2cbebe8558.
The old links are broken

I changed the links and created the new dataset_infos.json.

Pinging @thomwolf @lhoestq for notification.


KILT dataset: empty string in triviaqa input field

Just found out about https://github.com/huggingface/datasets/blob/master/datasets/kilt_tasks/README.md
(Not very clear in https://huggingface.co/datasets/kilt_tasks links to http://github.com/huggingface/datasets/datasets/kilt_tasks/README.md which is dead, closing the issue though :))

# What happened
Both train and test splits of the triviaqa dataset (part of the KILT benchmark) seem to have empty string in their input field (unlike the natural questions dataset, part of the same benchmark)

# Versions
KILT version is `1.0.0`
`datasets` version is `1.1.2`
[more here](https://gist.github.com/PaulLerner/3768c8d25f723edbac20d99b6a4056c1)

# How to reproduce
```py
In [1]: from datasets import load_dataset
In [4]: dataset = load_dataset("kilt_tasks")
# everything works fine, removed output for a better readibility
Dataset kilt_tasks downloaded and prepared to /people/lerner/.cache/huggingface/datasets/kilt_tasks/all_tasks/1.0.0/821c4295a2c35db2847585918d9c47d7f028f1a26b78825d8e77cd3aeb2621a1. Subsequent calls will reuse this data.

# empty string in triviaqa input field
In [36]: dataset['train_triviaqa'][0]
Out[36]:
{'id': 'dpql_5197',
 'input': '',
 'meta': {'left_context': '',
  'mention': '',
  'obj_surface': {'text': []},
  'partial_evidence': {'end_paragraph_id': [],
   'meta': [],
   'section': [],
   'start_paragraph_id': [],
   'title': [],
   'wikipedia_id': []},
  'right_context': '',
  'sub_surface': {'text': []},
  'subj_aliases': {'text': []},
  'template_questions': {'text': []}},
 'output': {'answer': ['five  £', '5 £', '£5', 'five £'],
  'meta': [],
  'provenance': [{'bleu_score': [1.0],
    'end_character': [248],
    'end_paragraph_id': [30],
    'meta': [],
    'section': ['Section::::Question of legal tender.\n'],
    'start_character': [246],
    'start_paragraph_id': [30],
    'title': ['Banknotes of the pound sterling'],
    'wikipedia_id': ['270680']}]}}
In [35]: dataset['train_triviaqa']['input'][:10]
Out[35]: ['', '', '', '', '', '', '', '', '', '']
# same with test set
In [37]: dataset['test_triviaqa']['input'][:10]
Out[37]: ['', '', '', '', '', '', '', '', '', '']
# works fine with natural questions
In [34]: dataset['train_nq']['input'][:10]
Out[34]:
['how i.met your mother who is the mother',
 'who had the most wins in the nfl',
 'who played mantis guardians of the galaxy 2',
 'what channel is the premier league on in france',
 "god's not dead a light in the darkness release date",
 'who is the current president of un general assembly',
 'when do the eclipse supposed to take place',
 'what is the name of the sea surrounding dubai',
 'who holds the nba record for most points in a career',
 'when did the new maze runner movie come out']
```

Stay safe :)


add amazon reviews

@patrickvonplaten Yeah this is adapted from tfds so a lot is just how they wrote the code. Addressed your comments and also simplified the weird `AmazonUSReviewsConfig` definition. Will merge once tests pass.

Thanks for checking this one :)
Looks good to me

Just one question : is there a particular reason to use `names=["Y", "N"]` in this order ? Usually the positive label is at index 1 and the negative one at index 0 for binary classification

> is there a particular reason to use `names=["Y", "N"]` in this order ? Usually the positive label is at index 1 and the negative one at index 0 for binary classification

Hmm that's a good point. I'll submit a quick fix.

Adds the Amazon US Reviews dataset as requested in #353. Converted from [TensorFlow Datasets](https://www.tensorflow.org/datasets/catalog/amazon_us_reviews). cc @clmnt @sshleifer


Error running pip install -e ".[dev]" on MacOS 10.13.6: faiss/python does not exist

I saw that `faiss-cpu` 1.6.4.post2 was released recently to fix the installation on macos. It should work now

Closing this one.
Feel free to re-open if you still have issues

I was following along with https://huggingface.co/docs/datasets/share_dataset.html#adding-tests-and-metadata-to-the-dataset when I ran into this error.

```sh
git clone https://github.com/huggingface/datasets
cd datasets
virtualenv venv -p python3 --system-site-packages
source venv/bin/activate
pip install -e ".[dev]"
```

![image](https://user-images.githubusercontent.com/59632/97868518-72871800-1cd5-11eb-9cd2-37d4e9d20b39.png)

![image](https://user-images.githubusercontent.com/59632/97868592-977b8b00-1cd5-11eb-8f3c-0c409616149c.png)

Python 3.7.7


dataset(ncslgr): add initial loading script

Hi @AmitMY, sorry for leaving you hanging for a minute :)

We've developed a new pipeline for adding datasets with a few extra steps, including adding a dataset card. You can find the full process [here](https://github.com/huggingface/datasets/blob/master/ADD_NEW_DATASET.md)

Would you be up for adding the tags and description in the README.md so we can merge this cool dataset?

@lhoestq should be ready for another review :)

Awesome thank you !

It looks like the PR now includes changes from other PR that were previously merged.
Feel free to create another branch and another PR so that we can have a clean diff.

Its a small dataset, but its heavily annotated
https://www.bu.edu/asllrp/ncslgr.html

![image](https://user-images.githubusercontent.com/5757359/97838609-3c539380-1ce9-11eb-885b-a15d4c91ea49.png)


failed to reuse cache

I packed the `load_dataset ` in a function of class, and cached data in a directory. But when I import the class and use the function, the data still have to be downloaded again. The information (Downloading and preparing dataset cnn_dailymail/3.0.0 (download: 558.32 MiB, generated: 1.28 GiB, post-processed: Unknown size, total: 1.82 GiB) to ******) which logged to terminal shows the path is right to the cache directory, but the files still have to be downloaded again.


Adding nli_tr dataset

Thank you @lhoestq for the time you take to review our pull request. We appreciate your help.

We've made the changes you described. Hope that it is ready for being merged. Please let me know if you have any additional requests for revisions.

Hello,

In this pull request, we have implemented the necessary interface to add our recent dataset [NLI-TR](https://github.com/boun-tabi/NLI-TR).  The datasets will be presented on a full paper at EMNLP 2020 this month. [[arXiv link] ](https://arxiv.org/pdf/2004.14963.pdf)

The dataset is the neural machine translation of SNLI and MultiNLI datasets into Turkish.  So, we followed a similar format with the original datasets hosted in the HuggingFace datasets hub.

Our dataset is designed to be accessed as follows by following the interface of the GLUE dataset that provides multiple datasets in a single interface over the HuggingFace datasets hub.

```
from datasets import load_dataset
multinli_tr = load_dataset("nli_tr", "multinli_tr")
snli_tr = load_dataset("nli_tr", "snli_tr")
```

Thanks for your help in reviewing our pull request.


feat(dataset): multiprocessing _generate_examples

I agree that would be cool :)
Right now the only distributed dataset builder is based on Apache Beam so you can use distributed processing frameworks like Dataflow, Spark, Flink etc. to build your dataset but it's not really well suited for single-worker parallel processing afaik

`_generate_examples` can now be run in parallel thanks to https://github.com/huggingface/datasets/pull/5107. You can find more info [here](https://huggingface.co/docs/datasets/dataset_script#sharding).

forking this out of #741, this issue is only regarding multiprocessing

I'd love if there was a dataset configuration parameter `workers`, where when it is `1` it behaves as it does right now, and when its `>1` maybe `_generate_examples` can also get the `pool` and return an iterable using the pool.

In my use case, I would instead of:
```python
for datum in data:
     yield self.load_datum(datum)
```
do:
```python
return pool.map(self.load_datum, data)
```

As the dataset in question, as an example, has **only** 7000 rows, and takes 10 seconds to load each row on average, it takes almost 20 hours to load the entire dataset.
If this was a larger dataset (and many such datasets exist), it would take multiple days to complete.

Using multiprocessing, for example, 40 cores, could speed it up dramatically. For this dataset, hopefully to fully load in under an hour.


feat(aslg_pc12): add dev and test data splits

Hi ! I'm not sure we should make this split decision arbitrarily on our side. Users can split it afterwards to whatever they want using `dataset.train_test_split` for example.
Moreover it looks like there's already papers that use this dataset and propose their own splits ([here](http://xanthippi.ceid.upatras.gr/HealthSign/resources/Publications/sitis_paper_25_10.pdf) 80-20)
What do you think ?

I was not aware of the `train_test_split` method, thanks!
Soe ven though it contributes to reproducibility, no need to do this split then.

For reproducibility sake, it's best if there are defined dev and test splits.

The original paper author did not define splits for the entire dataset, not for the sample loaded via this library, so I decided to define:
- 5/7th for train
- 1/7th for dev
- 1/7th for test


Issue with downloading Wikipedia data for low resource language

Hello, maybe you could ty to use another date for the wikipedia dump (see the available [dates](https://dumps.wikimedia.org/jvwiki) here for `jv`) ?

Thanks for reporting I created a PR to make the custom config work (language="zh", date="20201120").

For posterity, here's how I got the data I needed:  I needed Bengali, so I had to check which dumps are available here: https://dumps.wikimedia.org/bnwiki/ , then I ran:
```
load_dataset("wikipedia", language="bn", date="20211101",
                        beam_runner="DirectRunner")
```

Hi, I tried to download Sundanese and Javanese wikipedia data with the following snippet
```
jv_wiki = datasets.load_dataset('wikipedia', '20200501.jv', beam_runner='DirectRunner')
su_wiki = datasets.load_dataset('wikipedia', '20200501.su', beam_runner='DirectRunner')
```
And I get the following error for these two languages:
Javanese
```
FileNotFoundError: Couldn't find file at https://dumps.wikimedia.org/jvwiki/20200501/dumpstatus.json
```

Sundanese
```
FileNotFoundError: Couldn't find file at https://dumps.wikimedia.org/suwiki/20200501/dumpstatus.json
```

I found from https://github.com/huggingface/datasets/issues/577#issuecomment-688435085 that for small languages, they are directly downloaded and parsed from the Wikipedia dump site, but both of `https://dumps.wikimedia.org/jvwiki/20200501/dumpstatus.json` and `https://dumps.wikimedia.org/suwiki/20200501/dumpstatus.json` are no longer valid.

 Any suggestions on how to handle this issue? Thanks!


updated links to v1.3 of quail, fixed the description

we're using quail 1.3 now thanks.
closing this one


Fix metric deletion when attribuets are missing

When you call `del` on a metric we want to make sure that the arrow attributes are not already deleted.
I just added `if hasattr(...)` to make sure it doesn't crash


Add XNLI train set

Hi! Thanks for adding the translated MNLI! Do you know what translations system / model you used when you created the datasets in the other languages?

According to the [paper](https://arxiv.org/pdf/1809.05053.pdf) it's the result of the work of professional translators ;)

Thanks for getting back to me.

The training data is not from translators. And it appears to be machine
translation for all languages. If we can know what system was used to
create the training data that would be great!

Yifan.

On Thu, Jun 9, 2022, 05:51 Quentin Lhoest ***@***.***> wrote:

> According to the paper <https://arxiv.org/pdf/1809.05053.pdf> it's the
> result of the work of professional translators ;)
>
> —
> Reply to this email directly, view it on GitHub
> <https://github.com/huggingface/datasets/pull/781#issuecomment-1150914429>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AAKLKWDAPTMGB6BE5GJ4GULVOG5BLANCNFSM4TE67NMQ>
> .
> You are receiving this because you commented.Message ID:
> ***@***.***>
>

> The training data is not from translators.

What makes you think that ? The paper litteraly says

> we hire translators to translate the resulting sentences into 15 languages using the One Hour Translation platform.

However the annotators only did test and validation sets, as this was what
in the paper: “we construct an evaluation set for XLU by extending the
development and test sets of the Multi-Genre Natural Language Inference
Corpus (MultiNLI) to 15 languages".

On Thu, Jun 9, 2022 at 10:35 AM Quentin Lhoest ***@***.***>
wrote:

> The training data is not from translators.
>
> What makes you think that ? The paper litteraly says
>
> we hire translators to translate the resulting sentences into 15 languages
> using the One Hour Translation platform.
>
> —
> Reply to this email directly, view it on GitHub
> <https://github.com/huggingface/datasets/pull/781#issuecomment-1151202195>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AAKLKWFZOQPLK4WSKFRLW6DVOH6LLANCNFSM4TE67NMQ>
> .
> You are receiving this because you commented.Message ID:
> ***@***.***>
>

I added the train set that was built using the translated MNLI.
Now you can load the dataset specifying one language:
```python
from datasets import load_dataset

xnli_en = load_dataset("xnli", "en")
print(xnli_en["train"][0])
# {'hypothesis': 'Product and geography are what make cream skimming work .', 'label': 1, 'premise': 'Conceptually cream skimming has two basic dimensions - product and geography .'}
print(xnli_en["test"][0])
# {'hypothesis': 'I havent spoken to him again.', 'label': 2, 'premise': "Well, I wasn't even thinking about that, but I was so frustrated, and, I ended up talking to him again."}
```

Cc @sgugger


Add ASNQ dataset

Very nice !
What do the `sentence1` and `sentence2` correspond to exactly ?
Also maybe you could use the `ClassLabel` feature type for the `label` field (see [snli](https://github.com/huggingface/datasets/blob/master/datasets/snli/snli.py) for example)

> What do the `sentence1` and `sentence2` correspond to exactly ?

`sentence1` is a question, and `sentence2` is a candidate answer sentence. The labels are [1, 2, 3, 4] defining a relation between the answer sentence and the question. For example, label 4 means that the answer sentence is inside the _long_answer_ passage AND that the _short_answer_ is within the answer sentence. All the other labels are the negatives with different characteristics. (the short_answer, long_answer terminology is borrowed from Google's NQ dataset)

Should I label them simply as `question` and `answer`? I was going more with what I saw in the examples/run_glue.py script, but I realize now there is no restriction around this.

> Also maybe you could use the `ClassLabel` feature type for the `label` field (see [snli](https://github.com/huggingface/datasets/blob/master/datasets/snli/snli.py) for example)

I am finding it difficult to assign names to each class, but perhaps it's possible. Here's the description of each class from the paper.

1. Sentences from the document that are in the long answer but do not contain the annotated short answers. It is possible that these sentences might contain the short answer.
2. Sentences from the document that are not in the long answer but contain the short answer string, that is, such occurrence is purely accidental.
3. Sentences from the document that are neither in the long answer nor contain the short answer.
4. Sentences from the document that are in the long answer and do contain the annotated short answers.

Any ideas?

Yes it's better to have explicit feature names. Maybe go with question/answer or question/sentence.
I read in the paper that 1,2 and 3 are considered negative and 4 positive.
We could have a binary classification label `label` (either positive of negative) and then two boolean fields `short_answser_in_sentence` and `sentence_in_long_answer`. What do you think ?

> Yes it's better to have explicit feature names. Maybe go with question/answer or question/sentence.
> I read in the paper that 1,2 and 3 are considered negative and 4 positive.
> We could have a binary classification label `label` (either positive of negative) and then two boolean fields `short_answser_in_sentence` and `sentence_in_long_answer`. What do you think ?

Ok, sounds good. I went with `sentence` to keep it consistent with `short_answer_in_sentence` and `sentence_in_long_answer`.

I changed it to a ClassLabel with pos and neg classes and added the two above as features. Let me know if this is not what you had in mind.

This pull request adds the ASNQ dataset. It is a dataset for answer sentence selection derived from Google Natural Questions (NQ) dataset (Kwiatkowski et al. 2019). The dataset details can be found in the paper at https://arxiv.org/abs/1911.04118

The dataset is authored by Siddhant Garg, Thuy Vu and Alessandro Moschitti.

_Please note that I have no affiliation with the authors._

Repo: https://github.com/alexa/wqa_tanda


Feature/fidelity metrics from emnlp2020 evaluating and characterizing human rationales

Hi ! This looks interesting, thanks for adding it :)

For metrics there should only be two features fields: references and predictions.
Both of them can be defined as you want using nested structures if you need to.
Also I'm not sure what goes into references and what goes into predictions, could you give more details please ?
All the other computations parameters (model etc.) are fine though. Maybe explain a bit more what they're used for

> Hi ! This looks interesting, thanks for adding it :)
>
> For metrics there should only be two features fields: references and predictions.
> Both of them can be defined as you want using nested structures if you need to.
> Also I'm not sure what goes into references and what goes into predictions, could you give more details please ?
> All the other computations parameters (model etc.) are fine though. Maybe explain a bit more what they're used for

The `predictions` are the predicted labels by a model for a particular input. Do you mean making `prob_y_hat` - the probability of the prediction being the predicted label, `prob_y_hat_alpha` - the probability of the prediction being the predicted label when the input is reduced subject to alpha and the `null_difference` is the difference between the probability of the prediction being the predicted label in full information minus the probability in zero information a part of references? Also, I have added the description for other parameters in kwargs_description. I can expand it if that makes sense?

I think every value that is generated by the model (so label, prob_y_hat, prob_y_hat_alpha etc.) should be in `predictions`.
Feel free to add more details in the kwargs_description, this is very useful for the end user.

Hi @lhoestq , I have updated the code according to your feedback. Please, let me know if it looks good and can be merged now.

Metrics are deprecated in `datasets` and `evaluate` should be used instead: https://github.com/huggingface/evaluate

This metric computes fidelity (Yu et al. 2019, DeYoung et al. 2019) and normalized fidelity (Carton et al. 2020).


Unexpected behavior when loading cached csv file?

Hi ! Thanks for reporting.
The same issue was reported in #730 (but with the encodings instead of the delimiter). It was fixed by #770 .
The fix will be available in the next release :)

Thanks for the prompt reply and terribly sorry for the spam!
Looking forward to the new release!

I read a csv file from disk and forgot so specify the right delimiter. When i read the csv file again specifying the right delimiter it had no effect since it was using the cached dataset. I am not sure if this is unwanted behavior since i can always specify `download_mode="force_redownload"`. But i think it would be nice if the information what `delimiter` or what `column_names` were used would influence the identifier of the cached dataset.

Small snippet to reproduce the behavior:
```python
import datasets

with open("dummy_data.csv", "w") as file:
    file.write("test,this;text\n")

print(datasets.load_dataset("csv", data_files="dummy_data.csv", split="train").column_names)
# ["test", "this;text"]

print(datasets.load_dataset("csv", data_files="dummy_data.csv", split="train", delimiter=";").column_names)
# still ["test", "this;text"]
```

By the way, thanks a lot for this amazing library! :)


Better error message for uninitialized metric

When calling `metric.compute()` without having called `metric.add` or `metric.add_batch` at least once, the error was quite cryptic. I added a better error message

Fix #729


Allow custom split names in text dataset

Awesome! This will make the behaviour much more intuitive for some non-standard code.

Thanks!

The `text` dataset used to return only splits like train, test and validation. Other splits were ignored.
Now any split name is allowed.

I did the same for `json`, `pandas` and `csv`

Fix #735


Properly delete metrics when a process is killed

Tests are flaky when using metrics in distributed setup.
There is because of one test that make sure that using two possibly incompatible metric computation (same exp id) either works or raises the right error.
However if the error is raised, all the processes of the metric are killed, and the open files (arrow + lock files) are not closed correctly. This causes PermissionError on Windows when deleting the temporary directory.

To fix that I added a `finally` clause in the function passed to multiprocess to properly close the files when the process exits.


[ROUGE] Add description to Rouge metric

Add information about case sensitivity to ROUGE.


Adding CC-100: Monolingual Datasets from Web Crawl Data

dataset request

Requesting to add a new dataset

These dataset files are no longer available.  https://data.statmt.org/cc-100/ files provided in this link are no longer available. Can anybody fix that issue?
@abhishekkrthakur @yjernite

Hi ! Can you open an issue to report this problem ? This will help keep track of the fix :)

## Adding a Dataset
- **Name:** CC-100: Monolingual Datasets from Web Crawl Data
- **Description:** https://twitter.com/alex_conneau/status/1321507120848625665
- **Paper:** https://arxiv.org/abs/1911.02116
- **Data:** http://data.statmt.org/cc-100/
- **Motivation:** A large scale multi-lingual language modeling dataset. Text is de-duplicated and filtered by how "Wikipedia-like" it is, hopefully helping avoid some of the worst parts of the common crawl.

Instructions to add a new dataset can be found [here](https://huggingface.co/docs/datasets/share_dataset.html).


Fix metric with cache dir

The cache_dir provided by the user was concatenated twice and therefore causing FileNotFound errors.
The tests didn't cover the case of providing `cache_dir=` for metrics because of a stupid issue (it was not using the right parameter).

I remove the double concatenation and I fixed the tests

Fix #728


Using `Dataset.map` with `n_proc>1` print multiple progress bars

Yes it allows to monitor the speed of each process. Currently each process takes care of one shard of the dataset.

At one point we can consider using streaming batches to a pool of processes instead of sharding the dataset in `num_proc` parts. At that point it will be easy to use only one progress bar

Hi @lhoestq, I am facing a similar issue, it is annoying when lots of progress bars are printed. Is there a way to turn off this behavior?

You can disable the progress bars with
```python
import datasets

datasets.disable_progress_bar()
```

When using `Dataset.map` with `n_proc > 1`, only one of the processes should print a progress bar (to make the output readable). Right now, `n_proc` progress bars are printed.


Fix custom builder caching

The cache directory of a dataset didn't take into account additional parameters that the user could specify such as `features` or any parameter of the builder configuration kwargs (ex: `encoding` for the `text` dataset).

To fix that, the cache directory name now has a suffix that depends on all of them.

Fix #730
Fix #750


How to choose proper download_mode in function load_dataset?

`download_mode=datasets.GenerateMode.FORCE_REDOWNLOAD` should work.
This makes me think we we should rename this to DownloadMode.FORCE_REDOWNLOAD. Currently that's confusing

Can we just use `features=...` in `load_dataset` for this @lhoestq?

Indeed you should use `features` in this case.
```python
features = Features({'text': Value('string'), 'label': Value('float32')})
dataset = load_dataset('csv', data_files=['sst_test.csv'], features=features)
```
Note that because of an issue with the caching when you change the features (see #750 ) you still need to specify the `FORCE_REDOWNLOAD ` flag. I'm working on a fix for this one

https://github.com/huggingface/datasets/issues/769#issuecomment-717837832
> This makes me think we we should rename this to DownloadMode.FORCE_REDOWNLOAD. Currently that's confusing

@lhoestq do you still think we should rename it?

It's no big deal, but since it can be confusing to users I think it's worth renaming it, and deprecate `GenerateMode` until `datasets` 2.0 at least. IMO it's confusing to have `download_mode=GenerateMode.something`

Hi, I am a beginner to datasets and I try to use datasets to load my csv file.
my csv file looks like this

```
text,label
"Effective but too-tepid biopic",3
"If you sometimes like to go to the movies to have fun , Wasabi is a good place to start .",4
"Emerges as something rare , an issue movie that 's so honest and keenly observed that it does n't feel like one .",5
```

First I try to use this command to load my csv file .

``` python
dataset=load_dataset('csv', data_files=['sst_test.csv'])
```

It seems good, but when i try to overwrite the convert_options to convert  'label' columns from int64 to float32 like this.

``` python
import pyarrow as pa
from pyarrow import csv
read_options = csv.ReadOptions(block_size=1024*1024)
parse_options = csv.ParseOptions()
convert_options = csv.ConvertOptions(column_types={'text': pa.string(), 'label': pa.float32()})
dataset = load_dataset('csv', data_files=['sst_test.csv'], read_options=read_options,
                       parse_options=parse_options, convert_options=convert_options)
```

It keeps the same:

```shell
Dataset(features: {'text': Value(dtype='string', id=None), 'label': Value(dtype='int64', id=None)}, num_rows: 2210)
```

I think this issue is caused by the parameter "download_mode" Default to REUSE_DATASET_IF_EXISTS because after I delete the cache_dir, it seems right.

Is it a bug? How to choose proper download_mode to avoid this issue?


Add a `lazy_map` method to `Dataset` and `DatasetDict`

enhancement

New feature or request

This is cool! I think some aspects to think about and decide in terms of API are:
- do we allow several methods (chained i guess)
- how do we inspect the currently set method(s)
- how do we control/reset them

The library is great, but it would be even more awesome with a `lazy_map` method implemented on `Dataset` and `DatasetDict`. This would apply a function on a give item but when the item is requested. Two use cases:

1. load image on the fly
2. apply a random function and get different outputs at each epoch (like data augmentation or randomly masking a part of a sentence for BERT-like objectives).


Add option for named splits when using ds.train_test_split

New feature or request

Yes definitely we should give more flexibility to control the name of the splits outputted by `train_test_split`.

Related is the very interesting feedback from @bramvanroy on how we should improve this method: https://discuss.huggingface.co/t/how-to-split-main-dataset-into-train-dev-test-as-datasetdict/1090/5

And in particular that it should advantageously be able to split in 3 splits as well instead of just 2 like we copied from sklearn.

### Feature Request 🚀

Can we add a way to name your splits when using the `.train_test_split` function?

In almost every use case I've come across, I have a `train` and a `test` split in my `DatasetDict`, and I want to create a `validation` split. Therefore, its kinda useless to get a `test` split back from `train_test_split`, as it'll just overwrite my real `test` split that I intended to keep.

### Workaround

this is my hack for dealin with this, for now :slightly_smiling_face:

```python
from datasets import load_dataset

ds = load_dataset('imdb')
ds['train'], ds['validation'] = ds['train'].train_test_split(.1).values()
```


[GEM] add DART data-to-text generation dataset

dataset request

Requesting to add a new dataset

Yup, closing! Haven't been keeping track of the solved issues during the sprint.

## Adding a Dataset
- **Name:** DART
- **Description:** DART consists of 82,191 examples across different domains with each input being a semantic RDF triple set derived from data records in tables and the tree ontology of the schema, annotated with sentence descriptions that cover all facts in the triple set.
- **Paper:** https://arxiv.org/abs/2007.02871v1
- **Data:** https://github.com/Yale-LILY/dart
- **Motivation:** the dataset will likely be included in the GEM benchmark

Instructions to add a new dataset can be found [here](https://huggingface.co/docs/datasets/share_dataset.html).


[GEM] Add DART data-to-text generation dataset

Requesting to add a new dataset

## Adding a Dataset
- **Name:** DART
- **Description:** DART consists of 82,191 examples across different domains with each input being a semantic RDF triple set derived from data records in tables and the tree ontology of the schema, annotated with sentence descriptions that cover all facts in the triple set.
- **Paper:** https://arxiv.org/abs/2007.02871v1
- **Data:** https://github.com/Yale-LILY/dart
- **Motivation:** It will likely be included in the GEM generation evaluation benchmark

Instructions to add a new dataset can be found [here](https://huggingface.co/docs/datasets/share_dataset.html).


Adding Issue Template for Dataset Requests

adding .github/ISSUE_TEMPLATE/add-dataset.md


Fixed errors in bertscore related to custom baseline

[bertscore version 0.3.6 ](https://github.com/Tiiiger/bert_score) added support for custom baseline files. This update added extra argument `baseline_path` to BERTScorer class as well as some extra boolean parameters `use_custom_baseline` in functions like `get_hash(model, num_layers, idf, rescale_with_baseline, use_custom_baseline)`.

This PR fix those matching errors in bertscore metric implementation.


[GEM] Add Czech Restaurant data-to-text generation dataset

Requesting to add a new dataset

- Paper: https://www.aclweb.org/anthology/W19-8670.pdf
- Data: https://github.com/UFAL-DSG/cs_restaurant_dataset
- The dataset will likely be part of the GEM benchmark


Downloaded datasets are not usable offline

Yes currently you need an internet connection because the lib tries to check for the etag of the dataset script online to see if you don't have it locally already.

If we add a way to store the etag/hash locally after the first download, it would allow users to first download the dataset with an internet connection, and still have it working without an internet connection.

I'll let you know when we add this feature.

I've been trying to use the IMDB dataset offline, but after downloading it and turning off the internet it still raises  an error from the ```requests``` library trying to reach for the online dataset.
Is this the intended behavior ?
(Sorry, I wrote the the first version of this issue while still on nlp 0.3.0).


Add meta-data to the HANS dataset

good first issue

Good for newcomers

The current version of the [HANS dataset](https://github.com/huggingface/datasets/blob/master/datasets/hans/hans.py) is missing the additional information provided for each example, including the sentence parses, heuristic and subcase.


(Load dataset failure) ConnectionError: Couldn’t reach https://raw.githubusercontent.com/huggingface/datasets/1.1.2/datasets/cnn_dailymail/cnn_dailymail.py

Are you running the script on a machine with an internet connection ?

Yes ,  I can browse the url through Google Chrome.

Does this HEAD request return 200 on your machine ?
```python
import requests
requests.head("https://raw.githubusercontent.com/huggingface/datasets/1.1.2/datasets/cnn_dailymail/cnn_dailymail.py")
```

If it returns 200, could you try again to load the dataset ?

Thank you very much for your response.
When I run
```
import requests
requests.head("https://raw.githubusercontent.com/huggingface/datasets/1.1.2/datasets/cnn_dailymail/cnn_dailymail.py")
```
It returns 200.

And I try again to load the dataset. I got the following errors again.

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\666666\AppData\Local\Programs\Python\Python38\lib\site-packages\datasets\load.py", line 608, in load_dataset
    builder_instance.download_and_prepare(
  File "C:\Users\666666\AppData\Local\Programs\Python\Python38\lib\site-packages\datasets\builder.py", line 475, in download_and_prepare
    self._download_and_prepare(
  File "C:\Users\666666\AppData\Local\Programs\Python\Python38\lib\site-packages\datasets\builder.py", line 531, in _download_and_prepare
    split_generators = self._split_generators(dl_manager, **split_generators_kwargs)
  File "C:\Users\666666\.cache\huggingface\modules\datasets_modules\datasets\cnn_dailymail\0128610a44e10f25b4af6689441c72af86205282d26399642f7db38fa7535602\cnn_dailymail.py", line 253, in _split_generators
    dl_paths = dl_manager.download_and_extract(_DL_URLS)
  File "C:\Users\666666\AppData\Local\Programs\Python\Python38\lib\site-packages\datasets\utils\download_manager.py", line 254, in download_and_extract
    return self.extract(self.download(url_or_urls))
  File "C:\Users\666666\AppData\Local\Programs\Python\Python38\lib\site-packages\datasets\utils\download_manager.py", line 175, in download
    downloaded_path_or_paths = map_nested(
  File "C:\Users\666666\AppData\Local\Programs\Python\Python38\lib\site-packages\datasets\utils\py_utils.py", line 224, in map_nested
    mapped = [
  File "C:\Users\666666\AppData\Local\Programs\Python\Python38\lib\site-packages\datasets\utils\py_utils.py", line 225, in <listcomp>
    _single_map_nested((function, obj, types, None, True)) for obj in tqdm(iterable, disable=disable_tqdm)
  File "C:\Users\666666\AppData\Local\Programs\Python\Python38\lib\site-packages\datasets\utils\py_utils.py", line 163, in _single_map_nested
    return function(data_struct)
  File "C:\Users\666666\AppData\Local\Programs\Python\Python38\lib\site-packages\datasets\utils\file_utils.py", line 300, in cached_path
    output_path = get_from_cache(
  File "C:\Users\666666\AppData\Local\Programs\Python\Python38\lib\site-packages\datasets\utils\file_utils.py", line 475, in get_from_cache
    raise ConnectionError("Couldn't reach {}".format(url))
ConnectionError: Couldn't reach https://drive.google.com/uc?export=download&id=0BwmD_VLjROrfTHk4NFg2SndKcjQ

Connection error happened but the url was different.

I add the following code.
```
requests.head("https://drive.google.com/uc?export=download&id=0BwmD_VLjROrfTHk4NFg2SndKcjQ")
```
This  didn't  return 200
It returned like this:

Traceback (most recent call last):
  File "C:\Users\666666\AppData\Local\Programs\Python\Python38\lib\site-packages\urllib3\connection.py", line 159, in _new_conn
    conn = connection.create_connection(
  File "C:\Users\666666\AppData\Local\Programs\Python\Python38\lib\site-packages\urllib3\util\connection.py", line 84, in create_connection
    raise err
  File "C:\Users\666666\AppData\Local\Programs\Python\Python38\lib\site-packages\urllib3\util\connection.py", line 74, in create_connection
    sock.connect(sa)
TimeoutError: [WinError 10060]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\666666\AppData\Local\Programs\Python\Python38\lib\site-packages\urllib3\connectionpool.py", line 670, in urlopen
    httplib_response = self._make_request(
  File "C:\Users\666666\AppData\Local\Programs\Python\Python38\lib\site-packages\urllib3\connectionpool.py", line 381, in _make_request
    self._validate_conn(conn)
  File "C:\Users\666666\AppData\Local\Programs\Python\Python38\lib\site-packages\urllib3\connectionpool.py", line 978, in _validate_conn
    conn.connect()
  File "C:\Users\666666\AppData\Local\Programs\Python\Python38\lib\site-packages\urllib3\connection.py", line 309, in connect
    conn = self._new_conn()
  File "C:\Users\666666\AppData\Local\Programs\Python\Python38\lib\site-packages\urllib3\connection.py", line 171, in _new_conn
    raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPSConnection object at 0x000001F6060618E0>: Failed to establish a new connection: [WinError 10060]

Is google drive blocked on your network ?
For me
```python
requests.head("https://drive.google.com/uc?export=download&id=0BwmD_VLjROrfTHk4NFg2SndKcjQ")
```
returns 200

I can browse the google drive through google chrome. It's weird. I can download the dataset through google drive manually.

Could you try to update `requests` maybe ?
It works with 2.23.0 on my side

My ```requests``` is 2.24.0 . It still can't return 200.

Is it possible I download the dataset manually from google drive and use it for further test ? How can I do this ? I want to reproduce the model in this link https://huggingface.co/patrickvonplaten/bert2bert-cnn_dailymail-fp16. But I can't download the dataset through load_dataset method . I have tried many times  and the connection error always happens .

The head request should definitely work, not sure what's going on on your side.
If you find a way to make it work, please post it here since other users might encounter the same issue.

If you don't manage to fix it you can use `load_dataset` on google colab and then save it using `dataset.save_to_disk("path/to/dataset")`.
Then you can download the directory on your machine and do
```python
from datasets import load_from_disk
dataset = load_from_disk("path/to/local/dataset")
```

Hi
I want to know if this problem has been solved because I encountered a similar issue. Thanks.
`train_data = datasets.load_dataset("xsum", `split="train")`
`ConnectionError:` Couldn't reach https://raw.githubusercontent.com/huggingface/datasets/1.1.3/datasets/xsum/xsum.py`

Hi @smile0925 ! Do you have an internet connection ? Are you using some kind of proxy that may block the access to this file ?

Otherwise you can try to update `datasets` since we introduced retries for http requests in the 1.2.0 version
```
pip install --upgrade datasets
```
Let me know if that helps.

Hi @lhoestq
Oh, may be you are right. I find that my server uses some kind of proxy that block the access to this file.
![image](https://user-images.githubusercontent.com/46243662/106456211-2ca24180-64c8-11eb-831e-47e9b40e7da4.png)

> Hi @lhoestq
> Oh, may be you are right. I find that my server uses some kind of proxy that block the access to this file.
> ![image](https://user-images.githubusercontent.com/46243662/106456211-2ca24180-64c8-11eb-831e-47e9b40e7da4.png)

I have the same problem, have you solved it? Many thanks

Hi  @ZhengxiangShi
You can first try whether your network can access these files. I need to use VPN to access these files, so I download the files that cannot be accessed to the local in advance, and then use them in the code. Like this,
`train_data = datasets.load_dataset("xsum.py", split="train")`

For Ubuntu 20.04, there are the following feedback.

Google Drive is ok, but raw.githubusercontent.com has a big problem.  It seems that the raw github could not match the common urllib3 protocols.

**1. Google Drive**

```
import requests

requests.head("https://drive.google.com/uc?export=download&id=0BwmD_VLjROrfTHk4NFg2SndKcjQ")
<Response [200]>
```

**2.  raw.githubusercontent.com**

```
import requests
requests.head("https://raw.githubusercontent.com/huggingface/datasets/1.1.2/datasets/cnn_dailymail/cnn_dailymail.py")
```
........

raise CertificateError(
urllib3.util.ssl_match_hostname.CertificateError: hostname 'raw.githubusercontent.com' doesn't match either of 'default.ssl.fastly.net', 'fastly.com', '*.a.ssl.fastly.net', '*.hosts.fastly.net', '*.global.ssl.fastly.net', '*.fastly.com', 'a.ssl.fastly.net', 'purge.fastly.net', 'mirrors.fastly.net', 'control.fastly.net', 'tools.fastly.net'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
........
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='raw.githubusercontent.com', port=443): Max retries exceeded with url: /huggingface/datasets/1.1.2/datasets/cnn_dailymail/cnn_dailymail.py (Caused by SSLError(CertificateError("hostname 'raw.githubusercontent.com' doesn't match either of 'default.ssl.fastly.net', 'fastly.com', '*.a.ssl.fastly.net', '*.hosts.fastly.net', '*.global.ssl.fastly.net', '*.fastly.com', 'a.ssl.fastly.net', 'purge.fastly.net', 'mirrors.fastly.net', 'control.fastly.net', 'tools.fastly.net'")))

........

During handling of the above exception, another exception occurred:

Traceback (most recent call last):

.......

raise SSLError(e, request=request)
requests.exceptions.SSLError: HTTPSConnectionPool(host='raw.githubusercontent.com', port=443): Max retries exceeded with url: /huggingface/datasets/1.1.2/datasets/cnn_dailymail/cnn_dailymail.py (Caused by SSLError(CertificateError("hostname 'raw.githubusercontent.com' doesn't match either of 'default.ssl.fastly.net', 'fastly.com', '*.a.ssl.fastly.net', '*.hosts.fastly.net', '*.global.ssl.fastly.net', '*.fastly.com', 'a.ssl.fastly.net', 'purge.fastly.net', 'mirrors.fastly.net', 'control.fastly.net', 'tools.fastly.net'")))

**3. XSUM**

```
from datasets import load_dataset
raw_datasets = load_dataset("xsum", split="train")
```

ConnectionError: Couldn't reach https://raw.githubusercontent.com/EdinburghNLP/XSum/master/XSum-Dataset/XSum-TRAINING-DEV-TEST-SPLIT-90-5-5.json (SSLError(MaxRetryError('HTTPSConnectionPool(host=\'raw.githubusercontent.com\', port=443): Max retries exceeded with url: /EdinburghNLP/XSum/master/XSum-Dataset/XSum-TRAINING-DEV-TEST-SPLIT-90-5-5.json (Caused by SSLError(CertificateError("hostname \'raw.githubusercontent.com\' doesn\'t match either of \'default.ssl.fastly.net\', \'fastly.com\', \'*.a.ssl.fastly.net\', \'*.hosts.fastly.net\', \'*.global.ssl.fastly.net\', \'*.fastly.com\', \'a.ssl.fastly.net\', \'purge.fastly.net\', \'mirrors.fastly.net\', \'control.fastly.net\', \'tools.fastly.net\'")))')))

### The  following snippet could not solve the implicit ssl error.

```
import ssl

try:
    _create_unverified_https_context = ssl._create_unverified_context
except AttributeError:
    pass
else:
    ssl._create_default_https_context = _create_unverified_https_context
```

Only the oldest versions of `datasets` use raw.githubusercontent.com. Can you try updating `datasets` ?

Thank lhoestq fo the quick response.

I solve the big issue with the command line as follows.

**1. Open hosts (Ubuntu 20.04)**

`$ sudo gedit /etc/hosts`

**2. Add the command line into the hosts**

`151.101.0.133   raw.githubusercontent.com`

**3. Save hosts**

And then the jupyter notebook can access to the datasets (module)  and get the datasets of XSUM with raw.githubusercontent.com.

So it is not users' fault. But most of the suggestions in the web are wrong. Anyway, I solve the problem finally.

By the way, users need to add the other github commnads such as the following.

`199.232.69.194 github.global.ssl.fastly.net`

Cheers!!!

I use the dataset 2.14.4 that published on Aug 8, 2023.发自我的 iPhone在 2023年9月13日，06:38，Quentin Lhoest ***@***.***> 写道：
Only the oldest versions of datasets use raw.githubusercontent.com. Can you try updating datasets ?

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you commented.Message ID: ***@***.***>

Hey, I want to load the cnn-dailymail dataset for fine-tune.
I write the code like this
from datasets import load_dataset

test_dataset = load_dataset(“cnn_dailymail”, “3.0.0”, split=“train”)

And I got the following errors.

Traceback (most recent call last):
File “test.py”, line 7, in
test_dataset = load_dataset(“cnn_dailymail”, “3.0.0”, split=“test”)
File “C:\Users\666666\AppData\Local\Programs\Python\Python38\lib\site-packages\datasets\load.py”, line 589, in load_dataset
module_path, hash = prepare_module(
File “C:\Users\666666\AppData\Local\Programs\Python\Python38\lib\site-packages\datasets\load.py”, line 268, in prepare_module
local_path = cached_path(file_path, download_config=download_config)
File “C:\Users\666666\AppData\Local\Programs\Python\Python38\lib\site-packages\datasets\utils\file_utils.py”, line 300, in cached_path
output_path = get_from_cache(
File “C:\Users\666666\AppData\Local\Programs\Python\Python38\lib\site-packages\datasets\utils\file_utils.py”, line 475, in get_from_cache
raise ConnectionError(“Couldn’t reach {}”.format(url))
ConnectionError: Couldn’t reach https://raw.githubusercontent.com/huggingface/datasets/1.1.2/datasets/cnn_dailymail/cnn_dailymail.py

How can I fix this ?


Process 0 very slow when using num_procs with map to tokenizer

Hi ! Thanks for reporting.
Is the distribution of text length of your data evenly distributed across your dataset ? I mean, could it be because the examples in the first part of your dataset are slower to process ?
Also could how many CPUs can you use for multiprocessing ?
```python
import multiprocessing
print(multiprocessing.cpu_count())
```
Which tokenizer are you using ?

Using pre trained HF tokenizer. The result is the same with tokenizer multiprocessing off and on.
I have (absolutely) no idea about the distribution, but since this issue occurs on all of my datasets(regardless of files), I don't think distribution is the problems.

I can use up to 16 cores.

Ok weird, I don't manage to reproduce this issue on my side.
Does it happen even with `num_proc=2` for example ?
Also could you provide more details about your OS and the versions of tokenizers/datasets/multiprocess that you're using ?

Yes, I can confirm it also happens with ```num_proc=2```.
```
tokenizers               0.9.2
datasets                 1.1.2
multiprocess             0.70.10
```
```
Linux nipa2020-0629 4.4.0-178-generic #208-Ubuntu SMP Sun Apr 5 23:45:10 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
```

I can't reproduce on my side unfortunately with the same versions.

Do you have issues when doing multiprocessing with python ?
```python
from tqdm.auto import tqdm
from multiprocess import Pool, RLock

def process_data(shard):
    # implement

num_proc = 8
shards = [] # implement, this must be a list of size num_proc

with Pool(num_proc, initargs=(RLock(),), initializer=tqdm.set_lock) as pool:
    results = [pool.apply_async(process_data, shard=shard) for shard in shards]
    transformed_shards = [r.get() for r in results]
```

Nah, I'll just wait a few hours. Thank you for helping, though.

<img width="721" alt="image" src="https://user-images.githubusercontent.com/17930170/97066109-776d0d00-15ed-11eb-8bba-bb4d2e0fcc33.png">
The code I am using is
```

        dataset = load_dataset("text", data_files=[file_path], split='train')
        dataset = dataset.map(lambda ex: tokenizer(ex["text"], add_special_tokens=True,
                                                truncation=True, max_length=args.block_size), num_proc=8)
        dataset.set_format(type='torch', columns=['input_ids'])
        dataset.save_to_disk(file_path+'.arrow')
```


CUDA out of memory

Could you provide more details ? What's the code you ran ?

```python
tokenizer = FunnelTokenizer.from_pretrained('funnel-transformer/small')

def tokenize(batch):
    return tokenizer(batch['text'], padding='max_length', truncation=True,max_length=512)

dataset = load_dataset("bookcorpus",split='train[:1000]').shuffle()
dataset = dataset.map(tokenize, batched=True, batch_size=512)

# dataset = LineByLineTextDataset(
#     tokenizer=tokenizer,
#     file_path="./wiki1000.txt",
#     block_size=128
# )

data_collator = DataCollatorForLanguageModeling(
    tokenizer=tokenizer, mlm=True, mlm_probability=0.15
)

config=FunnelConfig(
    return_dict=True
)

model= FunnelForMaskedLM(config=config)

training_args = TrainingArguments(
    output_dir="./checkpoints",
    overwrite_output_dir=True,
    do_train=True,
    num_train_epochs=1,
    per_device_train_batch_size=16,
    per_device_eval_batch_size=16,
    save_steps=10000,
    logging_dir='./ptlogs'
)

trainer = Trainer(
    model=model,
    args=training_args,
    data_collator=data_collator,
    train_dataset=dataset,
)
trainer.train()
```

`RuntimeError: CUDA out of memory. Tried to allocate 954.00 MiB (GPU 0; 15.90 GiB total capacity; 14.35 GiB already allocated; 753.75 MiB free; 14.39 GiB reserved in total by PyTorch)
Exception raised from malloc at /pytorch/c10/cuda/CUDACachingAllocator.cpp:272 (most recent call first):`

part of error output

from funnel model to bert model : error still happened

from your dataset to LineByLineTextDataset : error disapeared

the error happens when executing loss.backward()

Since you're using a data collator you don't need to tokenizer the dataset using `map`. Could you try not to use `map` and only the data collator instead ? The data collator is supposed to pad to the longest sequence in each batch afaik, instead of padding to 512.

Also cc @sgugger

Closing this one.
Feel free to re-open if you have other questions about this issue

In your dataset ,cuda run out of memory as long as the trainer begins:
however, without changing any other element/parameter,just switch dataset to `LineByLineTextDataset`,everything becames OK.


Start community-provided dataset docs

Continuation of #736 with clean fork.

#### Old description
This is what I did to get the pseudo-labels updated. Not sure if it generalizes, but I figured I would write it down. It was pretty easy because all I had to do was make properly formatted directories and change URLs.

In slack @thomwolf called it a user-namespace dataset, but the docs call it community dataset.
I think the first naming is clearer, but I didn't address that here.

I didn't add metadata, will try that.


Start community-provided dataset docs V2


Use full released xsum dataset

@lhoestq I took a shot at addressing your comments but the build scripts seem to be complaining about not being able to open dummy files. How do I resolve those errors without copying the full dataset into the dummy dir?

Could you check that the names of the dummy data files are right ?
You can use
```
datasets-cli dummy_data ./datasets/xum
```
to print the expected file names

Ok @lhoestq looks like I got the tests to pass :)

#672 Fix xsum to expand coverage and include IDs
Code based on parser from older version of `datasets/xsum/xsum.py`
@lhoestq


Fix doc links to viewer

It seems #733 forgot some links in the doc :)


Clicking on a metric in the search page points to datasets page giving "Missing dataset" warning

Thanks for the report, can reproduce. Will fix

Hi! Sorry if this isn't the right place to talk about the website, I just didn't exactly where to write this.

Searching a metric in https://huggingface.co/metrics gives the right results but clicking on a metric (E.g ROUGE) points to https://huggingface.co/datasets/rouge. Clicking on a metric without searching points to the right page.

Thanks for all the great work!


Error loading ms_marco v2.1 using load_dataset()

There was a similar issue in #294
Clearing the cache and download again the dataset did the job. Could you try to clear your cache and download the dataset again ?

I was able to load the dataset successfully, I'm pretty sure it's just a cache issue that you have.
Let me know if clearing your cache fixes the problem

Yes, it indeed was a cache issue!
Thanks for reaching out!!

Code:
`dataset = load_dataset('ms_marco', 'v2.1')`

Error:
```
`---------------------------------------------------------------------------
JSONDecodeError                           Traceback (most recent call last)
<ipython-input-16-34378c057212> in <module>()
      9
     10 # Downloading and loading a dataset
---> 11 dataset = load_dataset('ms_marco', 'v2.1')

10 frames
/usr/lib/python3.6/json/decoder.py in raw_decode(self, s, idx)
    353         """
    354         try:
--> 355             obj, end = self.scan_once(s, idx)
    356         except StopIteration as err:
    357             raise JSONDecodeError("Expecting value", s, err.value) from None

JSONDecodeError: Unterminated string starting at: line 1 column 388988661 (char 388988660)
`
```


load_dataset doesn't include `features` in its hash

It looks like the function `load_dataset` does not include what's passed in the `features` argument when creating a hash for a given  dataset. As a result, if a user includes new features from an already downloaded dataset, those are ignored.

Example: some models on the hub have a different ordering for the labels than what `datasets` uses for MNLI so I'd like to do something along the lines of:
```
dataset = load_dataset("glue", "mnli")
features = dataset["train"].features
features["label"] = ClassLabel(names = ['entailment', 'contradiction', 'neutral'])  # new label order
dataset = load_dataset("glue", "mnli", features=features)
```


[XGLUE] Adding new dataset

dataset request

Requesting to add a new dataset

Small poll  @thomwolf @yjernite @lhoestq @JetRunner @qiweizhen .

As stated in the XGLUE paper: https://arxiv.org/pdf/2004.01401.pdf , for each of the 11 down-stream tasks training data is only available in English, whereas development and test data is available in multiple different language *cf.* here:

![Screenshot from 2020-11-04 15-02-17](https://user-images.githubusercontent.com/23423619/98120893-d7499a80-1eae-11eb-9d0b-57dfe5d4ee68.png)

So, I'd suggest to have exactly 11 "language-independent" configs: "ner", "pos", ... and give the sample in each dataset in the config a "language" label being one of "ar", "bg", .... => To me this makes more sense than making languaga specific config, *e.g.* "ner-de", ...especially because training data is only available in English. Do you guys agree?

In this case we should have named splits, so config `ner` has splits `train`, `validation`, `test-en`, `test-ar`, `test-bg`, etc...

This is more in the spirit of the task afaiu, and will avoid making users do the filtering step themselves when testing different models or different configurations of the same model.

I see your point!

I think this would be quite feasible to do and makes sense to me as well! In the paper results are reported per language, so it seems more natural to do it this way.

Good for me @yjernite ! What do the others think? @lhoestq

Okey actually not that easy to add things like `test-de` to `datasets` => this would be the first dataset to have this.
See: https://github.com/huggingface/datasets/pull/802

IMO we should have one config per language. That's what we're doing for xnli, xtreme etc.
Having split names that depend on the language seems wrong. We should try to avoid split names that are not train/val/test.
Sorry for late response on this one

@lhoestq agreed on having one config per language, but we also need to be able to have different split names and people are going to want to use hyphens, so we should at the very least warn them why it's failing :) E.g. for ANLI with different stages of data (currently using underscores) or https://www.tau-nlp.org/commonsenseqa with their train-sanity or dev-sanity splits

Yes sure ! Could you open a separate issue for that ?

Really cool dataset 👍 btw. does Transformers support all 11 tasks 🤔 would be awesome to have a xglue script (like the "normal" glue one)

Just to make sure this is what we want here. If we add one config per language,

this means that this dataset ends up with well over 100 different configs most of which will have the same `train` split. The train split is always in English. Also, I'm not sure whether it's better for the user to be honest.

I think it could be quite confusing for the user to have

```python
train_dataset = load_dataset("xglue", "ner-de", split="train")
```

in English even though it's `ner-de`.

To be honest, I'd prefer:

```python
train_dataset = load_dataset("xglue", "ner", split="train")
test_dataset_de = load_dataset("xglue", "ner", split="test-de")
test_dataset_fr = load_dataset("xglue", "ner", split="test-fr")
```

here

Oh yes right I didn't notice the train set was always in english sorry.
Moreover it seems that the way this dataset is used is to pick a pretrained multilingual model, fine-tune it on the english train set and then evaluate on each test set (one per language).
So to better fit the usual usage of this dataset, I agree that it's better to have one test split per language.

Something like your latest example patrick is fine imo :
```python
train_dataset = load_dataset("xglue", "ner", split="train")
test_dataset_de = load_dataset("xglue", "ner", split="test.de")
```

I just replace test-de with test.de since `-` is not allowed for split names (it has to follow the `\w+` regex), and usually we specify the language after a point.

Closing since XGLUE has been added in #802 , thanks patrick :)

I need xglue Urdu summarization dataset so how can i get it?

According to the table in https://huggingface.co/datasets/xglue, Urdu only exists for POS and XNLI in XGLUE - not for summarization

XGLUE is a multilingual GLUE like dataset propesed in this [paper](https://arxiv.org/pdf/2004.01401.pdf).

I'm planning on adding the dataset to the library myself in a couple of weeks.
Also tagging @JetRunner @qiweizhen in case I need some guidance


New version of CompGuessWhat?! with refined annotations

No worries. Always happy to help and thanks for your support in fixing the issue :)

This pull request introduces a few fixes to the annotations for VisualGenome in the CompGuessWhat?! original split.


Add Quail question answering dataset

QuAIL is a multi-domain RC dataset featuring news, blogs, fiction and user stories. Each domain is represented by 200 texts,    which gives us a 4-way data split. The texts are 300-350 word excerpts from CC-licensed texts that were hand-picked so as to make sense to human readers without larger context. Domain diversity mitigates the issue of possible overlap between training and test data of large pre-trained models, which the current SOTA systems are based on. For instance, BERT is trained on Wikipedia + BookCorpus, and was tested on Wikipedia-based SQuAD (Devlin, Chang, Lee, & Toutanova, 2019).

https://text-machine-lab.github.io/blog/2020/quail/ @annargrs


dataset(ngt): add ngt dataset initial loading script

Currently only making the paths to the annotation ELAN (eaf) file and videos available.
This is the first accessible way to download this dataset, which is not manual file-by-file.

Only downloading the necessary files, the annotation files are very small, 20MB for all of them, but the video files are large, 100GB in total, saved in `mpg` format.
I do not intend to actually store these as an uncompressed array of frames, because it will be huge.

Future updates may add pose estimation files for all videos, making it easier to work with this data


Fix emotion description

Hello, I probably have a silly question but the labels of the emotion dataset are in the form of numbers and not string, so I can not use the function classification_report because it mixes numbers and string (prediction). How can I access the label in the form of a string and not a number?
Thank you in advance.

Fixes the description of the emotion dataset to reflect the class names observed in the data, not the ones described in the paper.

I also took the liberty to make use of `ClassLabel` for the emotion labels.


Dataset Explorer Doesn't Work for squad_es and squad_it

Oups wrong click.
This one is for you @srush

https://huggingface.co/nlp/viewer/?dataset=squad_es
https://huggingface.co/nlp/viewer/?dataset=squad_it

Both pages show "OSError: [Errno 28] No space left on device".