`
san_yun
  • 浏览: 2651611 次
  • 来自: 杭州
文章分类
社区版块
存档分类
最新评论

CIFAR dataset

 
阅读更多

< Back to Alex Krizhevsky's home page :http://www.cs.toronto.edu/~kriz/

The CIFAR-10 and CIFAR-100 are labeled subsets of the 80 million tiny images dataset. They were collected by Alex Krizhevsky, Vinod Nair, and Geoffrey Hinton.

The CIFAR-10 dataset

The CIFAR-10 dataset consists of 60000 32x32 colour images in 10 classes, with 6000 images per class. There are 50000 training images and 10000 test images. 

The dataset is divided into five training batches and one test batch, each with 10000 images. The test batch contains exactly 1000 randomly-selected images from each class. The training batches contain the remaining images in random order, but some training batches may contain more images from one class than another. Between them, the training batches contain exactly 5000 images from each class. 

Here are the classes in the dataset, as well as 10 random images from each:

airplane
automobile
bird
cat
deer
dog
frog
horse
ship
truck


The classes are completely mutually exclusive. There is no overlap between automobiles and trucks. "Automobile" includes sedans, SUVs, things of that sort. "Truck" includes only big trucks. Neither includes pickup trucks.

Download

If you're going to use this dataset, please cite the tech report at the bottom of this page. 

Version Size md5sum
CIFAR-10 python version 163 MB c58f30108f718f92721af3b95e74349a
CIFAR-10 Matlab version 175 MB 70270af85842c9e89bb428ec9976c926
CIFAR-10 binary version (suitable for C programs) 162 MB c32a1d4ab5d03f1284b67883e8d87530

Baseline results

You can find some baseline replicable results on this dataset on the project page for cuda-convnet. These results were obtained with a convolutional neural network. Briefly, they are 18% test error without data augmentation and 11% with. Additionally, Jasper Snoek has a new paper in which he used Bayesian hyperparameter optimization to find nice settings of the weight decay and other hyperparameters, which allowed him to obtain a test error rate of 15% (without data augmentation) using the architecture of the net that got 18%.

Other results

Rodrigo Benenson has been kind enough to collect results on CIFAR-10/100 and other datasets on his website; click here to view.

Dataset layout

Python / Matlab versions

I will describe the layout of the Python version of the dataset. The layout of the Matlab version is identical. 

The archive contains the files data_batch_1data_batch_2, ..., data_batch_5, as well as test_batch. Each of these files is a Python "pickled" object produced with cPickle. Here is a Python routine which will open such a file and return a dictionary:

def unpickle(file):
    import cPickle
    fo = open(file, 'rb')
    dict = cPickle.load(fo)
    fo.close()
    return dict

Loaded in this way, each of the batch files contains a dictionary with the following elements:

  • data -- a 10000x3072 numpy array of uint8s. Each row of the array stores a 32x32 colour image. The first 1024 entries contain the red channel values, the next 1024 the green, and the final 1024 the blue. The image is stored in row-major order, so that the first 32 entries of the array are the red channel values of the first row of the image.
  • labels -- a list of 10000 numbers in the range 0-9. The number at index i indicates the label of the ith image in the array data.


The dataset contains another file, called batches.meta. It too contains a Python dictionary object. It has the following entries:

  • label_names -- a 10-element list which gives meaningful names to the numeric labels in the labels array described above. For example, label_names[0] == "airplane",label_names[1] == "automobile", etc.

Binary version

The binary version contains the files data_batch_1.bindata_batch_2.bin, ..., data_batch_5.bin, as well as test_batch.bin. Each of these files is formatted as follows:

<1 x label><3072 x pixel>
...
<1 x label><3072 x pixel>

In other words, the first byte is the label of the first image, which is a number in the range 0-9. The next 3072 bytes are the values of the pixels of the image. The first 1024 bytes are the red channel values, the next 1024 the green, and the final 1024 the blue. The values are stored in row-major order, so the first 32 bytes are the red channel values of the first row of the image. 

Each file contains 10000 such 3073-byte "rows" of images, although there is nothing delimiting the rows. Therefore each file should be exactly 30730000 bytes long. 

There is another file, called batches.meta.txt. This is an ASCII file that maps numeric labels in the range 0-9 to meaningful class names. It is merely a list of the 10 class names, one per row. The class name on row i corresponds to numeric label i.

The CIFAR-100 dataset

This dataset is just like the CIFAR-10, except it has 100 classes containing 600 images each. There are 500 training images and 100 testing images per class. The 100 classes in the CIFAR-100 are grouped into 20 superclasses. Each image comes with a "fine" label (the class to which it belongs) and a "coarse" label (the superclass to which it belongs).
Here is the list of classes in the CIFAR-100:

Superclass Classes
aquatic mammals beaver, dolphin, otter, seal, whale
fish aquarium fish, flatfish, ray, shark, trout
flowers orchids, poppies, roses, sunflowers, tulips
food containers bottles, bowls, cans, cups, plates
fruit and vegetables apples, mushrooms, oranges, pears, sweet peppers
household electrical devices clock, computer keyboard, lamp, telephone, television
household furniture bed, chair, couch, table, wardrobe
insects bee, beetle, butterfly, caterpillar, cockroach
large carnivores bear, leopard, lion, tiger, wolf
large man-made outdoor things bridge, castle, house, road, skyscraper
large natural outdoor scenes cloud, forest, mountain, plain, sea
large omnivores and herbivores camel, cattle, chimpanzee, elephant, kangaroo
medium-sized mammals fox, porcupine, possum, raccoon, skunk
non-insect invertebrates crab, lobster, snail, spider, worm
people baby, boy, girl, man, woman
reptiles crocodile, dinosaur, lizard, snake, turtle
small mammals hamster, mouse, rabbit, shrew, squirrel
trees maple, oak, palm, pine, willow
vehicles 1 bicycle, bus, motorcycle, pickup truck, train
vehicles 2 lawn-mower, rocket, streetcar, tank, tractor


Yes, I know mushrooms aren't really fruit or vegetables and bears aren't really carnivores. 

Download

Version Size md5sum
CIFAR-100 python version 161 MB eb9058c3a382ffc7106e4002c42a8d85
CIFAR-100 Matlab version 175 MB 6a4bfa1dcd5c9453dda6bb54194911f4
CIFAR-100 binary version (suitable for C programs) 161 MB 03b5dce01913d631647c71ecec9e9cb8

Dataset layout

Python / Matlab versions

The python and Matlab versions are identical in layout to the CIFAR-10, so I won't waste space describing them here.

Binary version

The binary version of the CIFAR-100 is just like the binary version of the CIFAR-10, except that each image has two label bytes (coarse and fine) and 3072 pixel bytes, so the binary files look like this:

<1 x coarse label><1 x fine label><3072 x pixel>
...
<1 x coarse label><1 x fine label><3072 x pixel>

Indices into the original 80 million tiny images dataset

Sivan Sabato was kind enough to provide this file, which maps CIFAR-100 images to images in the 80 million tiny images dataset. Sivan Writes:

The file has 60000 rows, each row contains a single index into the tiny db,
where the first image in the tiny db is indexed "1". "0" stands for an image that is not from the tiny db.
The first 50000 lines correspond to the training set, and the last 10000 lines correspond
to the test set.

Reference

This tech report (Chapter 3) describes the dataset and the methodology followed when collecting it in much greater detail. Please cite it if you intend to use this dataset. 

分享到:
评论

相关推荐

    cifar10Dataset-master

    cifar10Dataset-master,数据制作教程,可使用与深度学习

    CIFAR-10 dataset

    CIFAR-10数据集是计算机视觉领域中一个广泛使用的图像分类基准,它由Alex Krizhevsky、Vinod Nair和Geoffrey Hinton等人在2009年创建,是研究深度学习和图像识别算法的重要工具。这个数据集包含60,000个32x32像素的...

    The_classification_of_the_CIFAR_dataset_is_carried_

    The_classification_of_the_CIFAR_dataset_is_carried_PyTorch-CIFAR

    cifar-10-dataset-- python versions

    The CIFAR-10 and CIFAR-100 are labeled subsets of the 80 million tiny images dataset. They were collected by Alex Krizhevsky, Vinod Nair, and Geoffrey Hinton. 官网下载速度过慢

    CIFAR-10 Dataset-数据集

    CIFAR-10 Dataset is provided by Canadian Institute for Advanced Research.本数据集由加拿大高级研究所提供。 cifair10_trainLabels.csv cifar10_train.zip cifar10_test.zip

    对The CIFAR-10 dataset数据集合的研究与分析

    1 The CIFAR-10 dataset 10个类别 ,6万张图片 5万训练 1万测试,分别分为data_batch_1, data_batch_2, ..., data_batch_5 test_batch

    cifar-10-batches-py.zip

    The CIFAR-10 and CIFAR-100 are labeled subsets of the 80 million tiny images dataset. They were collected by Alex Krizhevsky, Vinod Nair, and Geoffrey Hinton. The CIFAR-10 dataset The CIFAR-10 dataset...

    Dataset Card for CIFAR-10

    The CIFAR-10 dataset consists of 60000 32x32 colour images in 10 classes, with 6000 images per class. There are 50000 training images and 10000 test images. The dataset is divided into five training ...

    dataset-cifar-10.zip

    在处理Cifar-10数据集时,首先需要对压缩包"dataset-cifar-10.zip"进行解压。解压后,你会看到多个文件,包括: 1. `data_batch_1`到`data_batch_5`:这些文件包含了训练集的图片数据。每个文件都包含10,000张图片...

    png格式CIFAR10-DATA(cifar10-data)

    The CIFAR-10 dataset :http://www.cs.toronto.edu/~kriz/cifar.html 原来为bin格式,我全部转成了png图,让我们对数据集有很好的直观印象。资源太大,分开上传,该部分为训练集部分(data_batch_1~data_batch_5)。

    cifar100-datasets-caffe-master

    标题 "cifar100-datasets-caffe-master" 指的是一个与 Caffe 框架相关的项目,其中包含了 CIFAR-100 数据集的实现和处理代码。CIFAR-100 是一个广泛使用的计算机视觉数据集,用于训练和评估深度学习模型,特别是图像...

    CIFAR-10 图片格式数据集

    CIFAR-10 图片格式数据集,按 10 分类文件夹储存 https://github.com/cyizhuo/CIFAR-10-dataset

    cifar-10-python

    The CIFAR-10 dataset consists of 60000 32x32 colour images in 10 classes, with 6000 images per class. There are 50000 training images and 10000 test images. The dataset is divided into five training ...

    CIFAR-100 Dataset-数据集

    CIFAR-100 Dataset is provided by Canadian Institute for Advanced Research.本数据集由加拿大高级研究所提供。 cifar100_test.zip cifar100_valid.zip cifar100_train.zip

    CIFAR-100 图片格式数据集

    CIFAR-100 图片格式数据集是计算机视觉领域常用的一个小型图像数据集,由Alex Krizhevsky、Ilya Sutskever和Geoffrey Hinton于2009年创建。这个数据集主要用于训练和评估深度学习模型,尤其是在图像分类任务上。...

    cifar-100-python.zip

    通常,数据加载器库,如TensorFlow的tf.data或PyTorch的torch.utils.data.Dataset,都可以方便地处理这个数据集,以便于训练和验证过程。 总的来说,CIFAR-100数据集是推动深度学习领域发展的关键资源之一,它不仅...

    png格式CIFAR10-DATA(cifar10-data)(2)

    The CIFAR-10 dataset :http://www.cs.toronto.edu/~kriz/cifar.html 原来为bin格式,我全部转成了png图,让我们对数据集有很好的直观印象。资源太大,分开上传,该部分为测试集部分(test_batch、batches.meta.txt)...

    cifar10_分类Python_cifar10分类_zip_cifar10代码下载_cifar10_

    for epoch in range(10): # loop over the dataset multiple times running_loss = 0.0 for i, data in enumerate(trainloader, 0): inputs, labels = data optimizer.zero_grad() outputs = net(inputs) ...

    Python CIFAR-1 数据集 (cifar-10-python.tar.gz)

    2. **数据加载器**:使用像 PyTorch 的 `DataLoader` 或 TensorFlow 的 `tf.data.Dataset` 来批量加载和处理数据,这可以并行化数据读取,加速训练过程。 3. **模型构建**:根据任务需求,选择合适的 CNN 架构。...

Global site tag (gtag.js) - Google Analytics