@harrisonqian / Awesome › wiki / big-data / public-datasets
Public View

Public Datasets

repo: awesomedata/awesome-public-datasets
category: Big Data related: Data Science · Machine Learning · Data Engineering · Streaming

Awesome Public Datasets

.. image:: https://cdn.rawgit.com/sindresorhus/awesome/d7305f38d29fed78fa85652e3a63e154dd8e8829/media/badge.svg :alt: Awesome :target: https://github.com/sindresorhus/awesome

This is a list of topic-centric public data sources <https://github.com/awesomedata/awesome-public-datasets>_ in high quality. They are collected and tidied from blogs, answers, and user responses. Most of the data sets listed below are free, however, some are not. This project was incubated at OMNILab <https://github.com/OMNILab>, Shanghai Jiao Tong University during Xiaming Chen's Ph.D. studies. OMNILab is now part of the BaiYuLan Open AI community <https://github.com/Bai-Yu-Lan>. Other amazingly awesome lists can be found in sindresorhus's awesome <https://github.com/sindresorhus/awesome>_ list.

NOTICE: This repo is automatically generated by apd-core <https://github.com/awesomedata/apd-core/tree/master/core>. Please DO NOT modify this file directly. We have provided a new way to contribute to this repo <https://github.com/awesomedata/apd-core/blob/master/CONTRIBUTING.md>. Join <https://join.slack.com/t/awesomedataworld/shared_invite/zt-dllew5xy-PJYi~mWUdY3hupohbmVZsA>_ the slack community <https://awesomedataworld.slack.com>_ for an instant touch of HQ data updates.

.. |OK_ICON| image:: https://raw.githubusercontent.com/awesomedata/apd-core/master/deploy/ok-24.png .. |FIXME_ICON| image:: https://raw.githubusercontent.com/awesomedata/apd-core/master/deploy/fixme-24.png

  • |OK_ICON| I am well.
  • |FIXME_ICON| Please fix me.

.. contents:: Table of Contents

Agriculture

  • |OK_ICON| The global dataset of historical yields for major crops 1981–2016 - The Global Dataset of [...] <https://doi.pangaea.de/10.1594/PANGAEA.909132>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Agriculture/Global-dataset-of-historical-yields-for-major-crops.yml>_]

  • |OK_ICON| Hyperspectral benchmark dataset on soil moisture - This dataset was measured in a five-day [...] <https://doi.org/10.5281/zenodo.1227837>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Agriculture/Hyperspectral-Benchmark-Dataset-On-Soil-Moisture.yml>_]

  • |OK_ICON| Lemons quality control dataset - Lemon dataset has been prepared to investigate the [...] <https://[github](/@harrisonqian/awesome/wiki/development-environment/github).com/softwaremill/lemon-dataset>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Agriculture/Lemon-Dataset.yml>_]

  • |OK_ICON| Optimized Soil Adjusted Vegetation Index - The IDB is a tool for working with remote sensing [...] <https://www.indexdatabase.de/db/i-single.php?id=63>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Agriculture/Optimized Soil Adjusted Vegetation Index>_]

  • |FIXME_ICON| U.S. Department of [Agriculture](/@harrisonqian/awesome/wiki/miscellaneous/agriculture)'s Nutrient [Database](/@harrisonqian/awesome/wiki/databases/database) - USDA National Nutrient [Database](/@harrisonqian/awesome/wiki/databases/database) for [...] <https://fdc.nal.usda.gov/download-[datasets](/@harrisonqian/awesome/wiki/miscellaneous/datasets)>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Agriculture/U.S.-Department-of-Agricultures-Nutrient-Database.yml>_]

  • |OK_ICON| U.S. Department of [Agriculture](/@harrisonqian/awesome/wiki/miscellaneous/agriculture)'s PLANTS [Database](/@harrisonqian/awesome/wiki/databases/database) - The Complete PLANTS Checklist is nearly 7 [...] <https://plants.usda.gov/downloads>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Agriculture/U.S.-Department-of-Agricultures-PLANTS-Database.yml>_]

Architecture

  • |OK_ICON| Swiss Apartment Models - This dataset contains detailed data on 42,207 apartments (242,257 [...] <https://zenodo.org/record/7070952#.Y0mACy0RqO0>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Architecture/appartment-models.yml>_]

Biology

  • |OK_ICON| 1000 Genomes - The 1000 Genomes Project ran between 2008 and 2015, creating the largest [...] <https://www.internationalgenome.org/data>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Biology/1000-Genomes.yml>_]

  • |OK_ICON| ANHIR - Automatic Non-rigid Histological Image Registration (ANHIR) consists of 2D [...] <https://anhir.grand-challenge.org/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Biology/ANHIR.yml>_]

  • |OK_ICON| American Gut (Microbiome Project) - The American Gut project is the largest crowdsourced [...] <https://github.com/biocore/American-Gut>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Biology/American-Gut-Microbiome-Project.yml>_]

  • |OK_ICON| BCNB - There are WSIs of 1058 patients, part of tumor regions are annotated in WSIs. Except [...] <https://bupt-ai-cz.[github](/@harrisonqian/awesome/wiki/development-environment/github).io/BCNB/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Biology/BCNB.yml>_]

  • |OK_ICON| Broad Bioimage Benchmark Collection (BBBC) - The Broad Bioimage Benchmark Collection (BBBC) [...] <https://www.broadinstitute.org/bbbc>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Biology/Broad-Bioimage-Benchmark-Collection-BBBC.yml>_]

  • |OK_ICON| Broad Cancer Cell Line Encyclopedia (CCLE) <http://www.broadinstitute.org/ccle/home>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Biology/Broad-Cancer-Cell-Line-Encyclopedia-CCLE.yml>_]

  • |FIXME_ICON| CIMA - CIMA dataset includes images of 2D histological microscopy tissue slices. <https://cmp.felk.cvut.cz/~borovji3/?page=dataset>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Biology/CIMA.yml>_]

  • |FIXME_ICON| Cell Image Library - This library is a public and easily accessible resource [database](/@harrisonqian/awesome/wiki/databases/database) of [...] <http://www.cellimagelibrary.org/home>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Biology/Cell-Image-Library.yml>_]

  • |FIXME_ICON| Complete Genomics Public Data - A diverse data set of whole human genomes are freely [...] <https://completegenomics.mgiamericas.com/demodata>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Biology/Complete-Genomics-Public-Data.yml>_]

  • |OK_ICON| CytoImageNet - A large-scale dataset of microscopy images. Contains 890,737 total grayscale [...] <https://www.kaggle.com/stanleyhua/cytoimagenet>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Biology/CytoImageNet.yml>_]

  • |OK_ICON| EBI ArrayExpress - ArrayExpress Archive of Functional Genomics Data stores data from high- [...] <http://www.ebi.ac.uk/arrayexpress/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Biology/EBI-ArrayExpress.yml>_]

  • |OK_ICON| EBI Protein Data Bank in Europe - The [Electron](/@harrisonqian/awesome/wiki/platforms/electron) Microscopy Data Bank (EMDB) is a public [...] <https://www.ebi.ac.uk/emdb/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Biology/EBI-Protein-Data-Bank-in-Europe.yml>_]

  • |OK_ICON| ENCODE project - The Encyclopedia of DNA Elements (ENCODE) Consortium is an ongoing [...] <https://www.encodeproject.org>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Biology/ENCODE-project.yml>_]

  • |OK_ICON| [Electron](/@harrisonqian/awesome/wiki/platforms/electron) Microscopy Pilot Image Archive (EMPIAR) - EMPIAR, the [Electron](/@harrisonqian/awesome/wiki/platforms/electron) Microscopy Public [...] <http://www.ebi.ac.uk/pdbe/emdb/empiar/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Biology/Electron-Microscopy-Pilot-Image-Archive-EMPIAR.yml>_]

  • |OK_ICON| Ensembl Genomes <https://ensemblgenomes.org/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Biology/Ensembl-Genomes.yml>_]

  • |OK_ICON| Gene Expression Omnibus (GEO) - GEO is a public functional genomics data repository [...] <http://www.ncbi.nlm.nih.gov/geo/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Biology/Gene-Expression-Omnibus-GEO.yml>_]

  • |OK_ICON| Gene Ontology (GO) - GO annotation files <http://geneontology.org/docs/download-go-annotations/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Biology/Gene-Ontology-GO.yml>_]

  • |OK_ICON| Global Biotic Interactions (GloBI) <https://github.com/jhpoelen/eol-globi-data/wiki#accessing-species-interaction-data>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Biology/Global-Biotic-Interactions-GloBI.yml>_]

  • |OK_ICON| Harvard Medical School (HMS) LINCS Project - The Harvard Medical School (HMS) LINCS Center is [...] <http://lincs.hms.harvard.edu>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Biology/Harvard-Medical-School-LINCS-Project.yml>_]

  • |FIXME_ICON| Human Genome [Diversity](/@harrisonqian/awesome/wiki/health-and-social-science/diversity) Project - A group of scientists at Stanford University have [...] <http://www.hagsc.org/hgdp/files.html>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Biology/Human-Genome-Diversity-Project.yml>_]

  • |FIXME_ICON| Human Microbiome Project (HMP) - The HMP sequenced over 2000 reference genomes isolated from [...] <http://www.hmpdacc.org/reference_genomes/reference_genomes.php>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Biology/Human-Microbiome-Project-HMP.yml>_]

  • |OK_ICON| ICOS PSP Benchmark - The ICOS PSP benchmarks repository contains an adjustable real-world [...] <http://ico2s.org/[datasets](/@harrisonqian/awesome/wiki/miscellaneous/datasets)/psp_benchmark.html>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Biology/ICOS-PSP-Benchmark.yml>_]

  • |OK_ICON| International HapMap Project <http://hapmap.ncbi.nlm.nih.gov/downloads/index.html.en>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Biology/International-HapMap-Project.yml>_]

  • |OK_ICON| Journal of Cell Biology DataViewer - All JCB data was moved to Biostudies <https://www.ebi.ac.uk/biostudies/JCB/studies>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Biology/Journal-of-Cell-Biology-DataViewer.yml>_]

  • |OK_ICON| KEGG - KEGG is a [database](/@harrisonqian/awesome/wiki/databases/database) resource for understanding high-level functions and utilities of [...] <http://www.genome.jp/kegg/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Biology/KEGG.yml>_]

  • |OK_ICON| NCBI Proteins <http://www.ncbi.nlm.nih.gov/guide/proteins/#databases>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Biology/NCBI-Proteins.yml>_]

  • |OK_ICON| NCBI Taxonomy - The NCBI Taxonomy [database](/@harrisonqian/awesome/wiki/databases/database) is a curated set of names and classifications for [...] <http://www.ncbi.nlm.nih.gov/taxonomy>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Biology/NCBI-Taxonomy.yml>_]

  • |OK_ICON| NCI Genomic Data Commons - The GDC Data Portal is a robust data-driven platform that allows [...] <https://gdc.cancer.gov/access-data/gdc-data-portal>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Biology/NCI-Genomic-Data-Commons.yml>_]

  • |OK_ICON| NIH Microarray data <ftp://ftp.ncbi.nih.gov/pub/geo/DATA/supplementary/series/GSE6532/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Biology/NIH-Microarray-data.yml>_]

  • |FIXME_ICON| OpenSNP genotypes data - openSNP allows customers of direct-to-customer genetic tests to [...] <https://opensnp.org/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Biology/OpenSNP-genotypes-data.yml>_]

  • |OK_ICON| Palmer Penguins - The goal of palmerpenguins is to provide a great dataset for data [...] <https://allisonhorst.[github](/@harrisonqian/awesome/wiki/development-environment/github).io/palmerpenguins/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Biology/Palmer-Penguins.yml>_]

  • |FIXME_ICON| Pathguid - Protein-Protein Interactions Catalog <http://www.pathguide.org/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Biology/Pathguid.yml>_]

  • |OK_ICON| Protein Data Bank - This resource is powered by the Protein Data Bank archive-information [...] <http://www.rcsb.org/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Biology/Protein-Data-Bank.yml>_]

  • |OK_ICON| Psychiatric Genomics Consortium - The purpose of the Psychiatric Genomics Consortium (PGC) is [...] <https://www.med.unc.edu/pgc/downloads>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Biology/Psychiatric-Genomics-Consortium.yml>_]

  • |OK_ICON| PubChem Project - PubChem is the world's largest collection of freely accessible chemical [...] <https://pubchem.ncbi.nlm.nih.gov/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Biology/PubChem-Project.yml>_]

  • |OK_ICON| PubGene (now Coremine Medical) - COREMINE™ is a family of tools developed by the Norwegian [...] <https://www.coremine.com/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Biology/PubGene-now-Coremine-Medical.yml>_]

  • |OK_ICON| Sanger Catalogue of Somatic Mutations in Cancer (COSMIC) - COSMIC, the Catalogue Of Somatic [...] <http://cancer.sanger.ac.uk/cosmic>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Biology/Sanger-Catalogue-of-Somatic-Mutations-in-Cancer-COSMIC.yml>_]

  • |FIXME_ICON| Sanger Genomics of Drug Sensitivity in Cancer Project (GDSC) <http://www.cancerrxgene.org/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Biology/Sanger-Genomics-of-Drug-Sensitivity-in-Cancer-Project-GDSC.yml>_]

  • |OK_ICON| Sequence Read Archive(SRA) - The Sequence Read Archive (SRA) stores raw sequence data from [...] <http://www.ncbi.nlm.nih.gov/Traces/sra/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Biology/Sequence-Read-ArchiveSRA.yml>_]

  • |OK_ICON| Serratus - Analysis of 7.1 million RNA/DNA sequencing [datasets](/@harrisonqian/awesome/wiki/miscellaneous/datasets) to discover the total [...] <https://github.com/ababaian/serratus/wiki/Access-Data-Release>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Biology/Serratus-Open-Virome.yml>_]

  • |OK_ICON| Stanford Microarray Data (Retired NOW) <http://smd.princeton.edu/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Biology/Stanford-Microarray-Data.yml>_]

  • |FIXME_ICON| Stowers Institute Original Data Repository <http://www.stowers.org/research/publications/odr>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Biology/Stowers-Institute-Original-Data-Repository.yml>_]

  • |OK_ICON| Systems Science of Biological Dynamics (SSBD) [Database](/@harrisonqian/awesome/wiki/databases/database) - Systems Science of Biological [...] <http://ssbd.qbic.riken.jp>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Biology/Systems-Science-of-Biological-Dynamics-SSBD-Database.yml>_]

  • |OK_ICON| The Cancer Genome Atlas (TCGA), available via Broad GDAC <https://gdac.broadinstitute.org/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Biology/The-Cancer-Genome-Atlas-TCGA-available-via-Broad-GDAC.yml>_]

  • |OK_ICON| The Catalogue of Life - The Catalogue of Life is a quality-assured checklist of more than 1.8 [...] <https://www.catalogueoflife.org/data/download>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Biology/The-Catalogue-of-Life.yml>_]

  • |OK_ICON| The Personal Genome Project - The Personal Genome Project, initiated in 2005, is a vision and [...] <http://www.personalgenomes.org/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Biology/The-Personal-Genome-Project.yml>_]

  • |OK_ICON| UCSC Public Data <http://hgdownload.soe.ucsc.edu/downloads.html>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Biology/UCSC-Public-Data.yml>_]

  • |OK_ICON| UniGene <https://ftp.ncbi.nlm.nih.gov/repository/UniGene/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Biology/UniGene.yml>_]

  • |OK_ICON| Universal Protein Resource (UnitProt) - The Universal Protein Resource (UniProt) is a [...] <http://www.uniprot.org/downloads>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Biology/Universal-Protein-Resource.yml>_]

  • |OK_ICON| Rfam - The Rfam [database](/@harrisonqian/awesome/wiki/databases/database) is a collection of RNA families, each represented by multiple [...] <https://docs.rfam.org/en/latest/database.html>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Biology/rfam.yml>_]

Chemistry

  • |OK_ICON| Ionic Liquids [Database](/@harrisonqian/awesome/wiki/databases/database) - ILThermo <https://ilthermo.boulder.nist.gov>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Chemistry/ionicliquids.yml>_]

Climate+Weather

  • |FIXME_ICON| Actuaries Climate Index <http://actuariesclimateindex.org/data/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Climate+Weather/Actuaries-Climate-Index.yml>_]

  • |FIXME_ICON| Australian Weather - Updated webpage for Australian Weather data. <https://www.bom.gov.au/climate/data/index.shtml??zoom=1&lat=-26.9635&lon=133.4635&layers=B0000TFFFFFFFFFTFFFFFFFFFFFFFFFFTTT&dp=IDC10001&p_nccObsCode=201&p_display_type=dailyDataFile>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Climate+Weather/Australian-Weather.yml>_]

  • |FIXME_ICON| Aviation Weather Center - Consistent, timely and accurate weather information for the world [...] <https://aviationweather.gov/adds/dataserver>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Climate+Weather/Aviation-Weather-Center.yml>_]

  • |FIXME_ICON| Brazilian Weather - Historical data (In Portuguese) - Data related to climate and weather [...] <http://sinda.crn.inpe.br/PCD/SITE/novo/site/historico/index.php>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Climate+Weather/Brazilian-Weather.yml>_]

  • |FIXME_ICON| Several Climate [Datasets](/@harrisonqian/awesome/wiki/miscellaneous/datasets) - The C3S Climate Data Store (CDS) is a one-stop shop for [...] <https://cds.climate.copernicus.eu/cdsapp#!/home>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Climate+Weather/CDS.yml>_]

  • |FIXME_ICON| Canadian Meteorological Centre <http://weather.gc.ca/grib/index_e.html>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Climate+Weather/Canadian-Meteorological-Centre.yml>_]

  • |OK_ICON| Caravan - a dataset for large-sample hydrology - Caravan is an open community dataset of [...] <https://zenodo.org/record/7540792#.Y-QQGK3MKUn>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Climate+Weather/Caravan.yml>_]

  • |FIXME_ICON| Climate Data from UEA (updated monthly) <https://www.uea.ac.uk/web/groups-and-centres/climatic-research-unit/data>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Climate+Weather/Climate-Data-from-UEA-updated-monthly.yml>_]

  • |FIXME_ICON| Dutch Weather - The KNMI Data Center (KDC) portal provides access to KNMI data on weather, [...] <https://data.knmi.nl/datasets>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Climate+Weather/Dutch-Weather.yml>_]

  • |OK_ICON| European Climate Assessment & Dataset <https://www.ecad.eu/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Climate+Weather/European-Climate-Assessment-&-Dataset.yml>_]

  • |OK_ICON| German Climate Data Center <https://cdc.dwd.de/portal/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Climate+Weather/German-Meteorological-Service-CDC.yml>_]

  • |OK_ICON| Global Climate Data Since 1929 <http://en.tutiempo.net/climate>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Climate+Weather/Global-Climate-Data-Since-1929.yml>_]

  • |OK_ICON| [Charting](/@harrisonqian/awesome/wiki/front-end-development/charting) The Global Climate Change News Narrative 2009-2020 - These four [datasets](/@harrisonqian/awesome/wiki/miscellaneous/datasets) represent [...] <https://blog.gdeltproject.org/four-massive-datasets-[charting](/@harrisonqian/awesome/wiki/front-end-development/charting)-the-global-climate-change-news-narrative-2009-2020/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Climate+Weather/GlobalClimateChangeNewsNarrative2009-2020.yml>_]

  • |OK_ICON| NASA Global Imagery Browse Services <https://wiki.earthdata.nasa.gov/display/GIBS>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Climate+Weather/NASA-Global-Imagery-Browse-Services.yml>_]

  • |OK_ICON| Average city temperatures - Contains daily time series of average air temperatures in about [...] <https://www.fetchseries.com/climate/average-city-temperatures-fsr/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Climate+Weather/NEW_DATASET.yml>_]

  • |OK_ICON| NOAA Bering Sea Climate <http://www.beringclimate.noaa.gov/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Climate+Weather/NOAA-Bering-Sea-Climate.yml>_]

  • |OK_ICON| NOAA Climate [Datasets](/@harrisonqian/awesome/wiki/miscellaneous/datasets) <http://www.ncdc.noaa.gov/data-access/quick-links>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Climate+Weather/NOAA-Climate-Datasets.yml>_]

  • |FIXME_ICON| NOAA Realtime Weather Models <http://www.ncdc.noaa.gov/data-access/model-data/model-datasets/numerical-weather-prediction>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Climate+Weather/NOAA-Realtime-Weather-Models.yml>_]

  • |OK_ICON| NOAA SURFRAD Meteorology and Radiation [Datasets](/@harrisonqian/awesome/wiki/miscellaneous/datasets) <https://www.esrl.noaa.gov/gmd/grad/stardata.html>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Climate+Weather/NOAA-SURFRAD-Meteorology-and-Radiation-Datasets.yml>_]

  • |OK_ICON| Open-Meteo - Open-Source Weather API - Open-source weather API with free access for non- [...] <https://open-meteo.com>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Climate+Weather/Open-Meteo.yml>_]

  • |OK_ICON| The World Bank Open Data Resources for Climate Change <http://data.worldbank.org/developers/climate-data-api>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Climate+Weather/The-World-Bank-Open-Data-Resources-for-Climate-Change.yml>_]

  • |FIXME_ICON| UEA Climatic Research Unit <http://www.cru.uea.ac.uk/data>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Climate+Weather/UEA-Climatic-Research-Unit.yml>_]

  • |OK_ICON| WU Historical Weather Worldwide <https://www.wunderground.com/history/index.html>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Climate+Weather/WU-Historical-Weather-Worldwide.yml>_]

  • |OK_ICON| Wahington Post Climate Change - To analyze warming temperatures in the United States, The [...] <https://github.com/washingtonpost/data-2C-beyond-the-limit-usa>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Climate+Weather/Washington Post Climate Change.yml>_]

  • |OK_ICON| WorldClim - Global Climate Data <http://www.worldclim.org>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Climate+Weather/WorldClim.yml>_]

ComplexNetworks

  • |OK_ICON| AMiner Citation Network Dataset <http://aminer.org/citation>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//ComplexNetworks/AMiner-Citation-Network-Dataset.yml>_]

  • |OK_ICON| CrossRef DOI URLs <https://archive.org/details/doi-urls>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//ComplexNetworks/CrossRef-DOI-URLs.yml>_]

  • |OK_ICON| DBLP Citation dataset <https://kdl.cs.umass.edu/display/public/DBLP>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//ComplexNetworks/DBLP-Citation-dataset.yml>_]

  • |OK_ICON| DIMACS Road Networks Collection <http://www.dis.uniroma1.it/challenge9/download.shtml>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//ComplexNetworks/DIMACS-Road-Networks-Collection.yml>_]

  • |OK_ICON| NBER Patent Citations <http://nber.org/patents/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//ComplexNetworks/NBER-Patent-Citations.yml>_]

  • |OK_ICON| NIST complex networks data collection <http://math.nist.gov/~RPozo/complex_datasets.html>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//ComplexNetworks/NIST-complex-networks-data-collection.yml>_]

  • |FIXME_ICON| Network Repository with Interactive Exploratory Analysis Tools <http://networkrepository.com/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//ComplexNetworks/Network-Repository-with-Interactive-Exploratory-Analysis-Tools.yml>_]

  • |FIXME_ICON| Protein-protein interaction network <http://vlado.fmf.uni-lj.si/pub/networks/data/bio/Yeast/Yeast.htm>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//ComplexNetworks/Protein.yml>_]

  • |OK_ICON| PyPI and Maven Dependency Network <https://ogirardot.wordpress.com/2013/01/31/sharing-pypimaven-dependency-data/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//ComplexNetworks/PyPI-and-Maven-Dependency-Network.yml>_]

  • |OK_ICON| Scopus Citation [Database](/@harrisonqian/awesome/wiki/databases/database) <https://www.elsevier.com/solutions/scopus>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//ComplexNetworks/Scopus-Citation-Database.yml>_]

  • |OK_ICON| Small Network Data <http://www-personal.umich.edu/~mejn/netdata/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//ComplexNetworks/Small-Network-Data.yml>_]

  • |OK_ICON| Stanford GraphBase <https://cs.stanford.edu/~knuth/sgb.html>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//ComplexNetworks/Stanford-GraphBase-Steven-Skiena.yml>_]

  • |OK_ICON| Stanford Large Network Dataset Collection <http://snap.stanford.edu/data/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//ComplexNetworks/Stanford-Large-Network-Dataset-Collection.yml>_]

  • |FIXME_ICON| Stanford Longitudinal Network Data Sources <http://stanford.edu/group/sonia/dataSources/index.html>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//ComplexNetworks/Stanford-Longitudinal-Network-Data-Sources.yml>_]

  • |FIXME_ICON| The Koblenz Network Collection <http://konect.uni-koblenz.de/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//ComplexNetworks/The-Koblenz-Network-Collection.yml>_]

  • |OK_ICON| The Laboratory for Web Algorithmics (UNIMI) <http://law.di.unimi.it/datasets.php>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//ComplexNetworks/The-Laboratory-for-Web-Algorithmics-UNIMI.yml>_]

  • |OK_ICON| UCI Network Data Repository <https://networkdata.ics.uci.edu/resources.php>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//ComplexNetworks/UCI-Network-Data-Repository.yml>_]

  • |OK_ICON| UFL sparse matrix collection <http://www.cise.ufl.edu/research/sparse/matrices/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//ComplexNetworks/UFL-sparse-matrix-collection.yml>_]

  • |FIXME_ICON| WSU Graph [Database](/@harrisonqian/awesome/wiki/databases/database) <http://www.eecs.wsu.edu/mgd/gdb.html>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//ComplexNetworks/WSU-Graph-Database.yml>_]

  • |FIXME_ICON| Community Resource for Archiving Wireless Data At Dartmouth - Contains [datasets](/@harrisonqian/awesome/wiki/miscellaneous/datasets) of pcap files [...] <https://www.crawdad.org/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//ComplexNetworks/crawdad.yml>_]

  • |OK_ICON| [Steam](/@harrisonqian/awesome/wiki/learn/steam) Co-Review Network - 82,000-game [Steam](/@harrisonqian/awesome/wiki/miscellaneous/steam) catalog with a 48,000-node co-review network [...] <https://github.com/lukeslp/[steam](/@harrisonqian/awesome/wiki/miscellaneous/steam)-network-data>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//ComplexNetworks/steam-network-data.yaml>_]

ComputerNetworks

  • |OK_ICON| 3.5B Web Pages from CommonCrawl 2012 <http://www.bigdatanews.com/profiles/blogs/big-data-set-3-5-billion-web-pages-made-available-for-all-of-us>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//ComputerNetworks/3.5B-Web-Pages-from-CommonCrawl-2012.yml>_]

  • |OK_ICON| 53.5B Web clicks of 100K users in Indiana Univ. <http://cnets.indiana.edu/groups/nan/webtraffic/click-dataset/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//ComputerNetworks/53.5B-Web-clicks-of-100K-users-in-Indiana-Univ..yml>_]

  • |OK_ICON| CAIDA Internet [Datasets](/@harrisonqian/awesome/wiki/miscellaneous/datasets) <http://www.caida.org/data/overview/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//ComputerNetworks/CAIDA-Internet-Datasets.yml>_]

  • |FIXME_ICON| CRAWDAD Wireless [datasets](/@harrisonqian/awesome/wiki/miscellaneous/datasets) from Dartmouth Univ. <https://crawdad.cs.dartmouth.edu/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//ComputerNetworks/CRAWDAD-Wireless-datasets-from-Dartmouth-Univ..yml>_]

  • |FIXME_ICON| ClueWeb09 - 1B web pages <http://lemurproject.org/clueweb09/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//ComputerNetworks/ClueWeb09.yml>_]

  • |FIXME_ICON| ClueWeb12 - 733M web pages <http://lemurproject.org/clueweb12/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//ComputerNetworks/ClueWeb12.yml>_]

  • |OK_ICON| CommonCrawl Web Data over 7 years <http://commoncrawl.org/the-data/get-started/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//ComputerNetworks/CommonCrawl-Web-Data-over-7-years.yml>_]

  • |OK_ICON| Shopper Intent Prediction from Clickstream E‑Commerce Data with Minimal Browsing Information <https://github.com/coveooss/shopper-intent-prediction-nature-2020>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//ComputerNetworks/Coveo-Shopper-Intent-Prediction.yaml>_]

  • |FIXME_ICON| Criteo click-through data <http://labs.criteo.com/2015/03/criteo-releases-its-new-dataset/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//ComputerNetworks/Criteo-click-through-data.yml>_]

  • |OK_ICON| Internet-Wide Scan Data Repository <https://scans.io/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//ComputerNetworks/Internet-Wide-Scan-Data-Repository.yml>_]

  • |OK_ICON| MIRAGE-2019 - MIRAGE-2019 is a human-generated dataset for mobile traffic analysis with [...] <http://traffic.comics.unina.it/mirage/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//ComputerNetworks/MIRAGE-2019.yml>_]

  • |OK_ICON| Merklemap DNS records Dataset - Contains 4B+ DNS records accross 700 million unique [...] <https://www.merklemap.com/dns-records-[database](/@harrisonqian/awesome/wiki/databases/database)>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//ComputerNetworks/Merklemap-DNS-Records-dataset.yml>_]

  • |OK_ICON| OONI: Open Observatory of Network Interference - Internet censorship data <https://ooni.torproject.org/data/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//ComputerNetworks/OONI-Open-Observatory-of-Network-Interference.yml>_]

  • |OK_ICON| Open Mobile Data by MobiPerf <https://console.developers.google.com/storage/openmobiledata_public/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//ComputerNetworks/Open-Mobile-Data-by-MobiPerf.yml>_]

  • |FIXME_ICON| The Peer-to-Peer Trace Archive - Real-world measurements play a key role in studying the [...] <http://p2pta.ewi.tudelft.nl/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//ComputerNetworks/P2P-Trace-Archive.yml>_]

  • |OK_ICON| Rapid7 Sonar Internet Scans <https://sonar.labs.rapid7.com/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//ComputerNetworks/Rapid7-Sonar-Internet-Scans.yml>_]

  • |OK_ICON| UCSD Network Telescope, IPv4 /8 net <http://www.caida.org/projects/network_telescope/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//ComputerNetworks/UCSD-Network-Telescope-IPv4-slash8-net.yml>_]

CyberSecurity

  • |OK_ICON| CCCS-CIC-AndMal-2020 - The dataset includes 200K benign and 200K malware samples totalling to [...] <https://www.unb.ca/cic/datasets/andmal2020.html>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//CyberSecurity/CCCS-CIC-AndMal-2020.yml>_]

  • |OK_ICON| Traffic and Log Data Captured During a Cyber Defense Exercise - This dataset was acquired [...] <https://zenodo.org/record/3746129>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//CyberSecurity/Traffic-and-Log-Data-Captured-During-a-Cyber-Defense-Exercise.yml>_]

DataChallenges

  • |OK_ICON| AIcrowd Competitions <https://www.aicrowd.com/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//DataChallenges/AIcrowd-Competitions.yml>_]

  • |OK_ICON| Bruteforce [Database](/@harrisonqian/awesome/wiki/databases/database) <https://github.com/duyetdev/bruteforce-database>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//DataChallenges/Bruteforce-Database.yml>_]

  • |OK_ICON| Challenges in [Machine Learning](/@harrisonqian/awesome/wiki/computer-science/machine-learning) <http://www.chalearn.org/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//DataChallenges/Challenges-in-Machine-[Learning](/@harrisonqian/awesome/wiki/programming-languages/learning).yml>_]

  • |FIXME_ICON| CrowdANALYTIX dataX <http://data.crowdanalytix.com>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//DataChallenges/CrowdANALYTIX-dataX.yml>_]

  • |FIXME_ICON| D4D Challenge of Orange <http://www.d4d.orange.com/en/home>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//DataChallenges/D4D-Challenge-of-Orange.yml>_]

  • |OK_ICON| DrivenData Competitions for Social Good <http://www.drivendata.org/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//DataChallenges/DrivenData-Competitions-for-Social-Good.yml>_]

  • |OK_ICON| ICWSM Data Challenge (since 2009) <https://www.icwsm.org/2018/datasets/datasets/#obtaining>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//DataChallenges/ICWSM-Data-Challenge-since-2009.yml>_]

  • |OK_ICON| KDD Cup by Tencent 2012 <http://www.kddcup2012.org/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//DataChallenges/KDD-Cup-by-Tencent-2012.yml>_]

  • |OK_ICON| Kaggle Competition Data <https://www.kaggle.com/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//DataChallenges/Kaggle-Competition-Data.yml>_]

  • |OK_ICON| Localytics [Data Visualization](/@harrisonqian/awesome/wiki/miscellaneous/data-visualization) Challenge <https://github.com/localytics/data-viz-challenge>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//DataChallenges/Localytics-Data-Visualization-Challenge.yml>_]

  • |OK_ICON| Netflix Prize <https://www.kaggle.com/datasets/netflix-inc/netflix-prize-data>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//DataChallenges/Netflix-Prize.yml>_]

  • |OK_ICON| Space [Apps](/@harrisonqian/awesome/wiki/platforms/apps) Challenge <https://2015.spaceappschallenge.org>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//DataChallenges/Space-Apps-Challenge.yml>_]

  • |FIXME_ICON| Telecom Italia [Big Data](/@harrisonqian/awesome/wiki/big-data/big-data) Challenge <https://dandelion.eu/datamine/open-big-data/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//DataChallenges/Telecom-Italia-Big-Data-Challenge.yml>_]

  • |FIXME_ICON| TravisTorrent Dataset - MSR'2017 Mining Challenge <https://travistorrent.testroots.org/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//DataChallenges/TravisTorrent-Dataset.yml>_]

  • |FIXME_ICON| TunedIT - Data mining & [machine learning](/@harrisonqian/awesome/wiki/computer-science/machine-learning) data sets, [algorithms](/@harrisonqian/awesome/wiki/theory/algorithms), challenges <http://tunedit.org/challenges/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//DataChallenges/TunedIT.yml>_]

  • |FIXME_ICON| Yelp Dataset Challenge - The Yelp dataset is a subset of our businesses, reviews, and user [...] <http://www.yelp.com/dataset>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//DataChallenges/Yelp-Dataset-Challenge.yml>_]

EarthScience

  • |OK_ICON| 38-Cloud (Cloud Detection) - Contains 38 Landsat 8 scene images and their manually extracted [...] <https://github.com/SorourMo/38-Cloud-A-Cloud-Segmentation-Dataset>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//EarthScience/38-Cloud.yml>_]

  • |OK_ICON| AQUASTAT - Global water resources and uses <http://www.fao.org/nr/water/aquastat/data/query/index.html?lang=en>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//EarthScience/AQUASTAT.yml>_]

  • |OK_ICON| BODC - marine data of ~22K vars <https://www.bodc.ac.uk/data/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//EarthScience/BODC.yml>_]

  • |FIXME_ICON| EOSDIS - NASA's [earth](/@harrisonqian/awesome/wiki/miscellaneous/earth) observing system data <http://sedac.ciesin.columbia.edu/data/sets/browse>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//EarthScience/EOSDIS.yml>_]

  • |OK_ICON| [Earth](/@harrisonqian/awesome/wiki/miscellaneous/earth) Models <https://www.earthmodels.org/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//EarthScience/Earth-Models.yml>_]

  • |OK_ICON| Global Wind Atlas - The Global Wind Atlas is a free, web-based application developed to help [...] <https://globalwindatlas.info/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//EarthScience/Global-Wind-Atlas.yml>_]

  • |OK_ICON| Integrated Marine Observing System (IMOS) - roughly 30TB of ocean measurements <https://imos.aodn.org.au>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//EarthScience/Integrated-Marine-Observing-System-IMOS.yml>_]

  • |FIXME_ICON| Marinexplore - Open Oceanographic Data <http://marinexplore.org/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//EarthScience/Marinexplore.yml>_]

  • |OK_ICON| Alabama Real-Time Coastal Observing System <http://mymobilebay.com>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//EarthScience/MyMobileBay.yml>_]

  • |OK_ICON| National Estuarine Research Reserves System-Wide Monitoring Program - long-term estuarine [...] <http://nerrsdata.org>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//EarthScience/NERRS-SWMP.yml>_]

  • |OK_ICON| Oil and Gas Authority Open Data - The dataset covers 12,500 offshore wellbores, 5,000 seismic [...] <https://data-ogauthority.opendata.arcgis.com/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//EarthScience/Oil-and-Gas-Authority-UK.yml>_]

  • |OK_ICON| Radiance [GeoJSON](/@harrisonqian/awesome/wiki/miscellaneous/geojson) — Global Light Pollution - Global nighttime light pollution dataset derived [...] <https://archive.org/details/radiance-[geojson](/@harrisonqian/awesome/wiki/miscellaneous/geojson)-data-2025>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//EarthScience/Radiance-GeoJSON-Light-Pollution.yml>_]

  • |OK_ICON| Smithsonian Institution Global Volcano and Eruption [Database](/@harrisonqian/awesome/wiki/databases/database) <http://volcano.si.edu/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//EarthScience/Smithsonian-Institution-Global-Volcano-and-Eruption-Database.yml>_]

  • |OK_ICON| USGS Earthquake Archives <http://earthquake.usgs.gov/earthquakes/search/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//EarthScience/USGS-Earthquake-Archives.yml>_]

  • |OK_ICON| Wellhead Protection Area (protection zone) prediction using breakthrough curves - This [...] <https://www.kaggle.com/datasets/robustus/whpa-prediction>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//EarthScience/WHPA.yml>_]

Economics

  • |FIXME_ICON| AI Displacement Tracker - A structured dataset tracking 109 documented cases of AI-driven [...] <https://github.com/noahaust2/ai-displacement-tracker>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Economics/AI-Displacement-Tracker.yml>_]

  • |OK_ICON| Asian [Productivity](/@harrisonqian/awesome/wiki/work/productivity) Organization (APO) - The AEPM provides a graphic dashboard view of [...] <https://www.apo-tokyo.org/wedo/[productivity](/@harrisonqian/awesome/wiki/work/productivity)-measurement/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Economics/APO.yml>_]

  • |OK_ICON| ASEAN Stats - The ASEANstatsDataPortal was first launched in June 2018. The Portal is [...] <https://data.aseanstats.org/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Economics/ASEAN Stats.yml>_]

  • |OK_ICON| American Economic Association (AEA) <https://www.aeaweb.org/resources/data>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Economics/American-Economic-Association-AEA.yml>_]

  • |OK_ICON| Asian KLEMS - Asia KLEMS is an Asian regional research consortium to promote building [...] <http://www.asiaklems.net/data/archive.asp>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Economics/Asian KLEMS.yml>_]

  • |OK_ICON| Harvard Atlas of Economic Complexity - A [database](/@harrisonqian/awesome/wiki/databases/database) for people to explore global trade flows [...] <https://dataverse.harvard.edu/dataverse/atlas>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Economics/Atlas Economic Complexity.yml>_]

  • |OK_ICON| BIS Financial [Database](/@harrisonqian/awesome/wiki/databases/database) - The files contain the same data as in the BIS Statistics Explorer [...] <https://www.bis.org/statistics/full_data_sets.htm>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Economics/BIS Financial Database.yml>_]

  • |OK_ICON| Barro-Lee [Education](/@harrisonqian/awesome/wiki/back-end-development/education) Attainment - Barro-Lee Educational Attainment Data from 1950 to 2010. [...] <http://www.barrolee.com/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Economics/Barro Lee.yml>_]

  • |OK_ICON| CEPII [Database](/@harrisonqian/awesome/wiki/databases/database) - A database of the world economy, through its country and region profiles, in [...] <http://www.cepii.fr/CEPII/en/bdd_modele/bdd_modele.asp>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Economics/CEPII Database.yml>_]

  • |OK_ICON| EUKLEMS - EU KLEMS is an industry level, growth and [productivity](/@harrisonqian/awesome/wiki/work/productivity) research project. EU KLEMS [...] <https://euklems.eu/query/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Economics/EUKLEMS.yml>_]

  • |FIXME_ICON| Economic Freedom of the World Data <https://www.fraserinstitute.org/economic-freedom/dataset>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Economics/Economic-Freedom-of-the-World-Data.yml>_]

  • |OK_ICON| Historical National Accounts - The datahub on Comparative Historical National Accounts [...] <https://www.rug.nl/ggdc/historicaldevelopment/na/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Economics/Historical National Accounts.yml>_]

  • |OK_ICON| Historical MacroEconomic Statistics <http://www.historicalstatistics.org/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Economics/Historical-MacroEconomic-Statistics.yml>_]

  • |FIXME_ICON| INFORUM - Interindustry Forecasting at the University of Maryland <https://inforumecon.com/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Economics/INFORUM.yml>_]

  • |OK_ICON| DBnomics – the world's economic [database](/@harrisonqian/awesome/wiki/databases/database) - Aggregates hundreds of millions of time series [...] <https://db.nomics.world/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Economics/International-[Economics](/@harrisonqian/awesome/wiki/miscellaneous/economics)-Database.yml>_]

  • |OK_ICON| International Trade Statistics - The new link contains trade based on filtered search on the [...] <https://data.worldbank.org/topic/21>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Economics/International-Trade-Statistics.yml>_]

  • |FIXME_ICON| Internet Product Code [Database](/@harrisonqian/awesome/wiki/databases/database) <http://www.upcdatabase.com/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Economics/Internet-Product-Code-Database.yml>_]

  • |OK_ICON| Joint External Debt Data Hub <http://www.jedh.org/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Economics/Joint-External-Debt-Data-Hub.yml>_]

  • |FIXME_ICON| Jon Haveman International Trade Data Links <https://freit.org/TradeResources/TradeData.php>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Economics/Jon-Haveman-International-Trade-Data-Links.yml>_]

  • |FIXME_ICON| Latin America KLEMS - LAKLEMS is a technical cooperation project financed by the Inter- [...] <http://laklems.net/stats/result>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Economics/LA KLEMS.yml>_]

  • |OK_ICON| Long-Term [Productivity](/@harrisonqian/awesome/wiki/work/productivity) [Database](/@harrisonqian/awesome/wiki/databases/database) - The Long-Term Productivity database was created as a [...] <http://longtermproductivity.com/download.html>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Economics/Long-Term-Productivity-Database.yml>_]

  • |OK_ICON| Maddison Project [Database](/@harrisonqian/awesome/wiki/databases/database) - The Maddison Project Database provides information on comparative [...] <https://www.rug.nl/ggdc/historicaldevelopment/maddison/releases/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Economics/Maddison Project.yml>_]

  • |OK_ICON| National Transfer Accounts - The goal of the National Transfer Accounts (NTA) project is to [...] <https://ntaccounts.org/web/nta/show/Browse%20database#H-zfl0oo>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Economics/NTA.yml>_]

  • |FIXME_ICON| OpenCorporates [Database](/@harrisonqian/awesome/wiki/databases/database) of Companies in the World <https://opencorporates.com/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Economics/OpenCorporates-Database-of-Companies-in-the-World.yml>_]

  • |OK_ICON| Our World in Data <http://ourworldindata.org/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Economics/Our-World-in-Data.yml>_]

  • |FIXME_ICON| Penn World Table - PWT version 10.0 is a [database](/@harrisonqian/awesome/wiki/databases/database) with information on relative levels of [...] <https://www.rug.nl/ggdc/productivity/pwt/?lang=en/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Economics/Penn World Table.yml>_]

  • |OK_ICON| Property Comps — Comparable Property Sales Across 11 Global Markets - Free API providing 44M+ [...] <https://api.nwc-advisory.com/docs>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Economics/Property-Comps.yml>_]

  • |OK_ICON| SciencesPo World Trade Gravity [Datasets](/@harrisonqian/awesome/wiki/miscellaneous/datasets) <http://econ.sciences-po.fr/thierry-mayer/data>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Economics/SciencesPo-World-Trade-Gravity-Datasets.yml>_]

  • |OK_ICON| Statistics of the World - Free API and interactive platform for 440+ economic, demographic, [...] <https://statisticsoftheworld.com/api-docs>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Economics/Statistics-of-the-World.yml>_]

  • |OK_ICON| The Atlas of Economic Complexity <http://atlas.cid.harvard.edu>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Economics/The-Atlas-of-Economic-Complexity.yml>_]

  • |FIXME_ICON| The Center for International Data <http://cid.econ.ucdavis.edu>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Economics/The-Center-for-International-Data.yml>_]

  • |OK_ICON| The Observatory of Economic Complexity <http://atlas.media.mit.edu/en/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Economics/The-Observatory-of-Economic-Complexity.yml>_]

  • |OK_ICON| UN Commodity Trade Statistics - The United Nations Comtrade [database](/@harrisonqian/awesome/wiki/databases/database) aggregates detailed [...] <https://comtradeplus.un.org/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Economics/UN-Commodity-Trade-Statistics.yml>_]

  • |OK_ICON| UN Human Development Reports <http://hdr.undp.org/en>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Economics/UN-Human-Development-Reports.yml>_]

  • |OK_ICON| World Input-Output [Database](/@harrisonqian/awesome/wiki/databases/database) - World Input-Output Tables and underlying data, covering 43 [...] <https://www.rug.nl/ggdc/valuechain/wiod/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Economics/World Input-Output Database.yml>_]

  • |OK_ICON| World KLEMS - Analytical KLEMS-type data sets for a broad set of countries around the world. [...] <https://www.worldklems.net/wkanalytical>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Economics/World KLEMS.yml>_]

  • |OK_ICON| Prop-Metrics — U.S. ZIP-Code-Level Real Estate [Analytics](/@harrisonqian/awesome/wiki/miscellaneous/analytics) Dataset - A comprehensive dataset [...] <https://www.prop-metrics.com>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Economics/prop-metrics-real-estate-data.yml>_]

Education

  • |FIXME_ICON| College Scorecard Data <https://collegescorecard.ed.gov/data/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Education/College-Scorecard-Data.yml>_]

  • |OK_ICON| New York State [Education](/@harrisonqian/awesome/wiki/back-end-development/education) Department Data - The New York State [Education](/@harrisonqian/awesome/wiki/back-end-development/education) Department (NYSED) is [...] <https://data.nysed.gov/downloads.php>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Education/New-York-State-Education-Department.yml>_]

  • |FIXME_ICON| Program for International Student Assessement (PISA) - Contains 15-year-old students' [...] <https://www.oecd.org/pisa/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Education/PISA.yml>_]

  • |OK_ICON| Student Data from Free Code Camp <https://github.com/freeCodeCamp/open-data>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Education/Student-Data-from-Free-Code-Camp.yml>_]

Energy

  • |OK_ICON| AMPds - The Almanac of Minutely Power dataset <http://ampds.org/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Energy/AMPds.yml>_]

  • |FIXME_ICON| BLUEd - Building-Level fUlly labeled Electricity Disaggregation dataset <https://energy.duke.edu/content/building-level-fully-labeled-electricity-disaggregation-blued>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Energy/BLUEd.yml>_]

  • |OK_ICON| COMBED <http://combed.[github](/@harrisonqian/awesome/wiki/development-environment/github).io/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Energy/COMBED.yml>_]

  • |OK_ICON| DBFC - Direct Borohydride Fuel Cell (DBFC) Dataset <https://github.com/ECSIM/dbfc-dataset>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Energy/DBFC.yml>_]

  • |OK_ICON| DEL - Domestic Electrical Load study datsets for South Africa (1994 - 2014) <https://www.datafirst.uct.ac.za/dataportal/index.php/catalog/DELS>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Energy/DEL.yml>_]

  • |OK_ICON| ECO - The ECO data set is a comprehensive data set for non-intrusive load monitoring and [...] <http://www.vs.inf.ethz.ch/res/show.html?what=eco-data>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Energy/ECO.yml>_]

  • |OK_ICON| EIA <http://www.eia.gov/electricity/data/eia923/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Energy/EIA.yml>_]

  • |OK_ICON| Global Power Plant [Database](/@harrisonqian/awesome/wiki/databases/database) - The Global Power Plant Database is a comprehensive, open source [...] <http://datasets.wri.org/dataset/globalpowerplantdatabase>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Energy/Global Power Plant Database.yml>_]

  • |OK_ICON| HES - Household Electricity Study, UK <http://randd.defra.gov.uk/Default.aspx?Menu=Menu&Module=More&Location=None&ProjectID=17359&FromSearch=Y&Publisher=1&SearchText=EV0702&SortString=ProjectCode&SortOrder=Asc&Paging=10#Description>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Energy/HES.yml>_]

  • |OK_ICON| HFED <http://hfed.[github](/@harrisonqian/awesome/wiki/development-environment/github).io/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Energy/HFED.yml>_]

  • |OK_ICON| MORED: a Moroccan Buildings’ Electricity Consumption Dataset - Since spring of 2019, a data [...] <https://github.com/MOREDataset/MORED>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Energy/MORED.yml>_]

  • |OK_ICON| Marktstammdatenregister - The German Marktstammdatenregister (MaStR) is a [database](/@harrisonqian/awesome/wiki/databases/database) of all [...] <https://www.marktstammdatenregister.de/MaStR/Datendownload>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Energy/MaStR.yml>_]

  • |OK_ICON| PEM1 - Proton Exchange Membrane (PEM) Fuel Cell Dataset <https://github.com/ECSIM/pem-dataset1>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Energy/PEM1.yml>_]

  • |FIXME_ICON| PLAID - The Plug Load Appliance Identification Dataset <http://plaidplug.com/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Energy/PLAID.yml>_]

  • |OK_ICON| The Public Utility Data Liberation Project (PUDL) - PUDL makes US energy data easier to [...] <https://github.com/catalyst-cooperative/pudl>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Energy/PUDL.yml>_]

  • |FIXME_ICON| REDD <http://redd.csail.mit.edu/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Energy/REDD.yml>_]

  • |OK_ICON| SYND - A synthetic energy dataset for non-intrusive load monitoring - With SynD, we present a [...] <https://www.nature.com/articles/s41597-020-0434-6>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Energy/SYND.yml>_]

  • |FIXME_ICON| Smart Meter Data Portal - The Smart Meter Data Portal is part of the National Science [...] <https://smda.[github](/@harrisonqian/awesome/wiki/development-environment/github).io/smart-meter-data-portal>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Energy/Smart Meter Data Portal.yml>_]

  • |OK_ICON| Tracebase <https://github.com/areinhardt/tracebase>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Energy/Tracebase.yml>_]

  • |OK_ICON| Ukraine Energy Centre [Datasets](/@harrisonqian/awesome/wiki/miscellaneous/datasets) <https://ukrstat.gov.ua/operativ/menu/menu_e/energ.htm>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Energy/UDEC.yml>_]

  • |OK_ICON| UK-DALE - UK Domestic Appliance-Level Electricity <https://jack-kelly.com/data>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Energy/UK-DALE.yml>_]

truncated — full list on GitHub

[[curator]]
I'm the Curator. I can help you navigate, organize, and curate this wiki. What would you like to do?