Social Media Data Hub#

Use Apify to collect social media data from TikTok, Instagram, X/Twitter, and YouTube through one consistent workflow.

Core Capabilities#

1. Profile lookup - Retrieve profile metadata such as follower count, content volume, and bio.
2. Bulk post retrieval - Pull recent or popular posts for an account together with engagement metrics.
3. Single-post lookup - Input a post URL and return a normalized record with key metrics.
4. Comment retrieval - Fetch comment threads for a supported post URL.
5. Cross-platform normalization - Map all supported platforms into one shared schema for analysis.

Actor Mapping#

PlatformActor IDPurposeTikTokclockworks/tiktok-scraperProfiles, posts, commentsInstagramapify/instagram-scraperProfiles, posts, commentsX/Twitter (bulk)apidojo/tweet-scraperBulk tweet retrieval (50+ items)X/Twitter (precise)apidojo/twitter-scraper-liteSingle tweet, conversation, or small batchesYouTubestreamers/youtube-scraperChannels and videos

Usage#

All scripts live under {baseDir}/scripts/, are executed with python3, and return JSON.

Profile Lookup#

bash

python3 {baseDir}/scripts/fetch_profile.py --platform tiktok --username "khaby.lame"
python3 {baseDir}/scripts/fetch_profile.py --platform instagram --username "natgeo"
python3 {baseDir}/scripts/fetch_profile.py --platform twitter --username "elonmusk"
python3 {baseDir}/scripts/fetch_profile.py --platform youtube --channel-url "https://www.youtube.com/@MrBeast"

Bulk Post Retrieval#

bash

python3 {baseDir}/scripts/fetch_posts.py --platform tiktok --username "khaby.lame" --count 20
python3 {baseDir}/scripts/fetch_posts.py --platform instagram --username "natgeo" --count 30
python3 {baseDir}/scripts/fetch_posts.py --platform twitter --username "elonmusk" --count 100
python3 {baseDir}/scripts/fetch_posts.py --platform youtube --channel-url "https://www.youtube.com/@MrBeast" --count 50

Single-Post Lookup#

bash

python3 {baseDir}/scripts/fetch_single_post.py --url "https://www.tiktok.com/@user/video/123456"
python3 {baseDir}/scripts/fetch_single_post.py --url "https://www.instagram.com/p/ABC123/"
python3 {baseDir}/scripts/fetch_single_post.py --url "https://x.com/user/status/123456"
python3 {baseDir}/scripts/fetch_single_post.py --url "https://www.youtube.com/watch?v=ABC123"

Comment Retrieval#

bash

python3 {baseDir}/scripts/fetch_comments.py --url "https://www.tiktok.com/@user/video/123456" --count 50
python3 {baseDir}/scripts/fetch_comments.py --url "https://www.instagram.com/p/ABC123/" --count 30
python3 {baseDir}/scripts/fetch_comments.py --url "https://x.com/user/status/123456" --count 100

Cross-Platform Normalization#

normalize.py is used internally by the fetch scripts and can also be run directly:

bash

echo '<raw_json>' | python3 {baseDir}/scripts/normalize.py --platform tiktok --type post

Unified Data Model#

Normalized Post ( NormalizedPost)#

FieldTypeDescriptionplatformstringtiktok / instagram / twitter / youtube``post_idstringPlatform-native post IDpost_urlstringPost URLtextstringCaption, text body, or titlecreated_atstringISO 8601 timestampauthor_namestringAuthor usernameauthor_display_namestringDisplay namelikesintLike countcommentsintComment countsharesintShare or repost count, or null``viewsintView count, play count, or null``savesintSave or bookmark count, or null``hashtagslistHashtag listmedia_typestringvideo / image / text / carousel

Normalized Profile ( NormalizedProfile)#

FieldTypeDescriptionplatformstringPlatform identifierusernamestringUsername or handledisplay_namestringDisplay namebiostringBiography or profile summaryfollowersintFollower countfollowingintFollowing count, or null``posts_countintTotal content count, or null``profile_urlstringProfile URLverifiedboolVerification status

Cost Reference (BRONZE Tier)#

OperationTikTokInstagramX/TwitterYouTubePer post$0.003$0.0023$0.0004$0.003Bulk 1K items$3.00$2.30$0.40$3.00Single-item lookup~$0.50 (minimum charge)$0.0023$0.05$0.003

See {baseDir}/references/apify_actors_reference.md for actor inputs, selection guidance, and pricing details.

Notes#

• TikTok has a $0.50 minimum per run, so bulk retrieval is usually more cost-effective than single-item lookups.
• For X/Twitter, use tweet-scraper when you need 50 or more items and twitter-scraper-lite for smaller batches or single tweets.
• Instagram comments are limited to 50 comments per post.
• YouTube comments are not handled by these scripts; use a dedicated YouTube Comments Scraper actor instead.
• All scripts call the Apify REST API directly and do not rely on the apify-client SDK.