Some of the most cited computer vision and machine learning datasets are built on YouTube video. Researchers building them face consistent questions: which content is ethically and legally usable, how do you document it for reproducibility, and how do you handle disappearing videos? A well-structured dataset from CC-licensed YouTube content is both academically defensible and technically achievable.
Working on a research project that involves YouTube video or audio data? TubePull handles individual video downloads and batch processing for Pro accounts — clean MP4s, MP3 audio, SRT caption files, and timestamped transcript exports, all from a browser interface. The rest of this guide covers dataset design, ethical review, metadata for reproducibility, and the companion tools that turn a list of URLs into a publishable research corpus.
The academic precedent for YouTube-based datasets
The most prominent example is YouTube-8M, a Google Research dataset containing millions of YouTube video IDs with machine-generated annotations. YouTube-8M distributes video IDs and pre-extracted features — not video files — because downloading and hosting millions of copyrighted videos would create copyright issues.
Distributing IDs and metadata rather than raw video files is the academic community's standard solution for large-scale YouTube corpora. Researchers reproduce the dataset by downloading videos corresponding to the published IDs; deleted or private videos simply fail gracefully at replication time.
Beyond YouTube-8M, YouTube has sourced:
- Computer vision benchmarks at CVPR, ICCV, and ECCV — action recognition (Kinetics dataset), video captioning, and object tracking.
- NLP and speech corpora — multilingual speech recognition training data, discourse analysis corpora, and interview transcription studies.
- Social science research — analysis of political communication, misinformation spread, and creator community dynamics.
- Computational linguistics — prosody, code-switching, non-native speaker patterns.
The through-line is consistent: CC-licensed content is the clean choice, and non-CC content needs a clear fair use or research exemption justification.
IRB and ethics review considerations
Academic video research involving human subjects requires ethics review, and YouTube datasets usually do involve human subjects.
Research involving public figures and publicly posted content may qualify for IRB exemption at many institutions when the unit of analysis is the video rather than individuals' identifying information.
Research involving comments, user behavior, or demographic inferences about individuals is more likely to require full review. Public content does not automatically exempt research into individuals' behavior.
Research involving vulnerable populations — minors, people in distress, communities with power differentials — requires careful review regardless.
Practical steps:
- Check your institution's IRB guidelines for internet research. Many have published specific guidance on social media and YouTube data.
- Consult your IRB coordinator before beginning data collection if you are unsure.
- Document your IRB determination (review approval number or exemption category) in your paper's methods section.
IRB clearance does not grant copyright permissions, and CC licensing does not substitute for IRB review when human subjects protections apply.
Filtering playlists for CC-licensed content
YouTube search with CC filter. Run a search, click Filters, and tick Creative Commons under Features. Save relevant URLs to a text file.
YouTube Data API. The YouTube Data API v3 supports programmatic search with videoLicense=creativeCommon. The free tier allows 10,000 units/day; each search costs 100 units.
Verifying individual videos. Even with the CC filter, verify each video's license in its description before including it in your dataset. License status can change after indexing.
License-based dataset documentation. Record the CC-BY license status of each video at collection time, along with the URL, video ID, uploader, title, and upload date. This documentation is the evidence that the dataset was legitimately sourced if the license later changes.
Metadata preservation for reproducibility
The core challenge of YouTube-based research is reproducibility: can another researcher reconstruct your dataset in five years? Given YouTube's deletion rate, the honest answer is "not fully" — but you can get close and make the gap explicit.
| Field | Why it matters |
|---|---|
| Video ID | The stable 11-character identifier. Use this rather than the full URL in published datasets. |
| Title at collection time | Titles can change after upload. |
| Channel name and channel ID | Human-readable name plus the stable /channel/UCxxxxxxx ID. |
| Upload date | As shown on YouTube. |
| Collection date | The "as-of" date for your metadata. |
| Description at collection time | Especially important if it confirms CC license status. |
| Video duration | If a replicator downloads a different-length video, they know something changed. |
| Resolution downloaded | Part of reproducibility for visual feature extraction. |
| Caption language and type | Manual vs. auto-generated; matters for NLP work. |
| SHA-256 hash of downloaded file | Proves the file is byte-identical to your version. |
TubePull batch download workflow
For individual videos and small batches (under fifty URLs), TubePull is the fastest manual workflow:
- Compile your URL list from the search and filter steps above.
- Open TubePull and paste each URL (free tier) or submit batches of five (Pro tier).
- Select MP4 at the appropriate resolution. 1080p is sufficient for most computer vision work; 720p saves storage at scale.
- Download to a consistently named local folder.
- Generate SHA-256 hashes for each file.
- Log all metadata fields in your dataset spreadsheet.
For larger batches, yt-dlp (see the yt-dlp vs GUI guide) handles hundreds of URLs from a text file using --batch-file. Combined with --write-info-json --write-description --write-thumbnail --write-subs, this produces a complete per-video metadata package alongside each video file.
TubePull gives you speed and simplicity for the filtered subset of verified-CC videos; yt-dlp gives you automation for large-scale collection.
Audio and transcript extraction for NLP datasets
Audio extraction. TubePull's MP3 format produces 320 kbps audio suitable for speech feature extraction, prosody analysis, and training speech models. The youtube-to-mp3-bitrate-explained guide covers audio quality considerations for research contexts.
Transcript extraction. TubePull's transcript export (plain text or Markdown with timestamps) converts caption tracks into research-ready text. For CC-licensed educational content, a batch of transcript extractions can constitute a substantial text corpus in hours.
SRT files for discourse analysis. TubePull exports raw SRT caption files for timestamped cue-level analysis. The download-creative-commons-youtube guide covers the license verification workflow that should precede bulk transcript collection.
Publishing a reproducible research dataset
- Publish the dataset card or data statement. Document collection methodology, date, license verification process, IRB status, and known limitations.
- Publish video IDs, not video files. Distributing downloaded videos likely violates copyright even for CC-BY content. Publish IDs; let readers reproduce.
- Publish your feature extractions if those are the actual data. Visual features, speech embeddings, or text tokens can be published directly without distributing the underlying videos.
- Include the expected video count and a deletion rate note. "As of [collection date], the dataset contained N videos. Replication attempts may encounter unavailable videos due to YouTube deletions or private status changes."
- Provide a download script. A Python script that takes your published ID list, checks each video's current status, and downloads available videos gives replicators a clean starting point.
How TubePull fits in
TubePull handles the individual-video and small-batch steps: clean MP4 files, MP3 audio, timestamped transcripts, and SRT caption files — without the installation overhead of command-line tools. For researchers verifying a few dozen URLs manually before building an automated pipeline, it is faster than any other option.
The broader academic workflow — CC license filtering via YouTube Data API, bulk download via yt-dlp for large datasets, metadata preservation, hash generation — uses TubePull for individual verification and spot-checking. Neither approach handles ethical and IRB questions; those are institutional obligations that precede any download.
Disclaimer. This article is general guidance for researchers and is not legal advice. Copyright rules for research datasets vary by jurisdiction, and IRB requirements vary by institution and country. The norms described here reflect common practice in the US academic research community; your institution's policies and your IRB's determinations govern your specific project. For commercial uses of research-derived datasets, consult an intellectual property attorney.