How to Scrape TikTok Videos, Profiles and Comments with an API

A public TikTok page looks simple: a username, a video, a comment thread. The collection contract is not. Profiles are not a video feed. Comments are not a full reply tree. Search is video-only. A live room lookup does not discover lives.

This tutorial builds a maintainable public-data workflow with the SocQ TikTok APIs. It covers endpoint selection, input hygiene, asynchronous tasks, normalized records, refresh cadence, and responsible use.

Quick answer: Route each public input to POST /v1/tiktok/{resource}, persist the returned task ID, and poll /v1/tasks/{id} until completion. Do not send a profile username to the videos endpoint, and do not treat a missing public field as zero.

Official TikTok API or a Public Data API?

TikTok's official Research, Display, and Content Posting products are the right choice when you have an approved research application, a login-authorized creator or advertiser workflow, or a need to publish. Those products use application review, OAuth, and contractual scopes. They are not a general scrape of arbitrary public videos.

The SocQ endpoints in this tutorial accept known public usernames, video URLs, queries, or hashtags and perform read-only collection. They do not authenticate as a TikTok user, grant rights to reuse the returned media, or replace official posting and research products. Choose the access model first: official APIs for authorized or approved work, a supported public-data API for narrowly scoped public reads.

Commerce listings live on a different surface. For product search and reviews, use the TikTok Shop APIs, not the social endpoints below.

Choose the Correct TikTok Endpoint

SocQ currently exposes twelve TikTok resources. This tutorial focuses on the six most common collection jobs. The rest exist and should be routed explicitly when you need them.

Also available, and easy to misuse if you guess:

• video-transcript for resolvable public video URLs.
• comment-replies for one video URL plus a known comment_id.
• trending-feed for a required region.
• followers-list and following-list for public usernames.
• live-room-info for a known room_id and user_id. It does not discover live rooms or wait for an account to go live.

These are public read operations. Compare provider models in TikTok scraper APIs and TikTok API alternatives when the job is vendor selection rather than implementation.

Normalize Inputs Before You Spend a Request

Store the raw input as provenance, then canonicalize a copy:

1. Strip a leading @ from usernames and a leading # from hashtags, or keep them — both forms are accepted, but your registry should use one canonical form.
2. Trim whitespace and reject empty values.
3. Convert profile URLs to usernames in your input layer. Convert share and mobile hosts to a www.tiktok.com/@user/video/{id} URL when the video ID is present.
4. Deduplicate usernames case-insensitively and URLs by canonical path.
5. Route by entity type. A username is not a video URL. A hashtag is not a search query.
6. Keep results_limit as a request cap, not a completeness guarantee.

user-videos accepts results_limit from 20 through 2000, sort_by of latest or popular, and an optional region. Search is video-only and accepts sort_by of relevance, date, or likes, plus published_within of day, week, month, three_months, or six_months.

Submit a Collection Task

Collect public profiles:

curl -X POST "https://api.socq.ai/v1/tiktok/profiles" \
  -H "Authorization: Bearer $SOCQ_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"usernames":["tiktok","@natgeo"]}'

Collect known videos:

curl -X POST "https://api.socq.ai/v1/tiktok/videos" \
  -H "Authorization: Bearer $SOCQ_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"urls":["https://www.tiktok.com/@tiktok/video/7234567890123456789"]}'

Collect a creator feed window:

curl -X POST "https://api.socq.ai/v1/tiktok/user-videos" \
  -H "Authorization: Bearer $SOCQ_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"usernames":["@tiktok"],"results_limit":20,"sort_by":"latest","region":"US"}'

Collect top-level comments, then replies only when you already have a comment ID:

curl -X POST "https://api.socq.ai/v1/tiktok/comments" \
  -H "Authorization: Bearer $SOCQ_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"urls":["https://www.tiktok.com/@tiktok/video/7234567890123456789"],"results_limit":100}'

curl -X POST "https://api.socq.ai/v1/tiktok/comment-replies" \
  -H "Authorization: Bearer $SOCQ_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"url":"https://www.tiktok.com/@scout2015/video/6718335390845095173","comment_id":"6718335906996502534","results_limit":20}'

Search and hashtag discovery use the same authentication and task envelope:

curl -X POST "https://api.socq.ai/v1/tiktok/search" \
  -H "Authorization: Bearer $SOCQ_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"query":"python tutorial","results_limit":50,"sort_by":"relevance","published_within":"month"}'

The response creates an asynchronous task. Persist its ID next to your internal batch ID and inputs.

Poll the Task without Creating Duplicates

curl "https://api.socq.ai/v1/tasks/$TASK_ID?limit=100" \
  -H "Authorization: Bearer $SOCQ_API_KEY"

Treat queued and running as expected states. Poll with exponential backoff and a maximum delay. For a successful task, process data.results.items; when has_more is true, continue with next_cursor.

Your job table should record:

• Internal batch ID and SocQ task ID.
• Resource type and canonical inputs.
• Submission, last poll, and completion timestamps.
• Terminal status and normalized error category.
• Last committed result cursor.

Do not resubmit merely because a worker restarted. Resume from persisted task state first.

Understand the Returned Fields

All records share normalized identity fields such as id, platform, resource, type, url, created_at, collected_at, and extra. Resource-specific fields remain explicit.

A profile includes username, name, description, avatar_url, is_verified, is_private, and visible metrics. It is not a list of that creator's videos.

A video includes caption, author, visible play/like/comment/share metrics, media, duration_seconds, hashtags, mentions, and music when public. Media URLs are source references, not permanent downloads.

A comment includes text, author, metrics.likes_count, and often metrics.replies_count. Reply counts are not reply text. Reply bodies require comment-replies and a comment_id from the same video.

Search and hashtag results reuse the video shape. They are ranked or windowed discovery results, not an exhaustive archive.

Missing data must remain null or absent. A hidden play count is not zero. An unavailable transcript is not an empty string you can treat as "no speech."

Python Worker for Mixed TikTok Inputs

import time
import requests

BASE = "https://api.socq.ai/v1"
HEADERS = {
    "Authorization": f"Bearer {SOCQ_API_KEY}",
    "Content-Type": "application/json",
}

def collect(resource, payload):
    created = requests.post(
        f"{BASE}/tiktok/{resource}",
        headers=HEADERS,
        json=payload,
        timeout=30,
    )
    created.raise_for_status()
    task_id = created.json()["data"]["task_id"]

    delay = 2
    while True:
        response = requests.get(
            f"{BASE}/tasks/{task_id}",
            headers=HEADERS,
            timeout=30,
        )
        response.raise_for_status()
        task = response.json()["data"]

        if task["status"] == "succeeded":
            return task["results"]["items"]
        if task["status"] in {"failed", "cancelled"}:
            raise RuntimeError(task)

        time.sleep(delay)
        delay = min(delay * 1.5, 15)

profiles = collect("profiles", {"usernames": ["tiktok"]})
videos = collect("user-videos", {
    "usernames": ["tiktok"],
    "results_limit": 20,
    "sort_by": "latest",
})

In production, persist the task before polling, paginate with next_cursor, and retry only transient failures.

Model Entities and Snapshots Separately

Profiles change slowly. Video metrics and comment volume change quickly.

tiktok_entity
  platform_id
  resource
  canonical_url_or_username
  stable public fields
  first_seen_at
  last_seen_at

tiktok_snapshot
  platform_id
  collected_at
  visible metrics
  mutable public fields
  source_task_id

Upsert the entity by resource + id when an ID exists. Fall back to a canonical URL or username and keep a merge path for later ID discovery. Append snapshots only after a successful collection and schema validation.

Do not calculate growth when either snapshot is missing the metric. A hidden count is not zero.

Validate Before Loading into Production

At minimum, verify:

• platform equals tiktok.
• resource matches the endpoint used.
• Profile records are not treated as a video feed.
• Comment records are top-level unless they came from comment-replies.
• Search results are videos, not users or sounds.
• Counts are non-negative when present.
• created_at and collected_at parse as timestamps.
• Every record can be traced to an input and task.

Quarantine invalid records rather than silently coercing them. Keep the raw response, validation error, and collection time.

Choose a Refresh Schedule by Entity

Use business need and data minimization to reduce unnecessary collection.

Common Failure Modes

The endpoint returns no item

Confirm the username or URL is public, still available, and routed to the correct resource. A private or deleted video should stay an input-level error, not an empty profile.

Profiles look "empty" of videos

That is expected. profiles returns profile records only. Use user-videos for a feed window.

Comments have a reply count but no reply text

The comments endpoint collects top-level comments. Fetch reply text with url plus comment_id. The replies endpoint does not discover comment_id from the video URL alone.

Search missed a sound or user

Search is video-only. Do not treat it as a user, sound, or LIVE directory.

Metrics do not match the browser

Counts are collection-time snapshots and may be rounded, hidden, cached, or updated between requests. Compare timestamps and field definitions.

Media URLs fail later

Treat video, cover, and avatar URLs as temporary source references. Refresh only when the use case and terms allow it.

Responsible TikTok Data Use

Collect only public fields needed for a legitimate purpose. Do not bypass login walls, private accounts, age gates, or technical restrictions. Avoid sensitive inference and automated decisions that could materially affect a person without appropriate safeguards.

TikTok data can be personal data. Establish a lawful basis where required, define retention and deletion procedures, secure API keys and stored records, honor applicable rights, and review TikTok's current terms and local law with qualified counsel. A public-data API does not grant extra rights to music, likeness, or video files.

The same catalog pattern is described in the unified social media API guide when TikTok is one source among many.

FAQ

Is scraping TikTok legal?

There is no universal answer. It depends on jurisdiction, data, access method, purpose, contract, and safeguards. This tutorial is not legal advice.

Are TikTok's official APIs free?

Official Research, Display, and Content Posting access is application-based or permissioned. This tutorial is a public-read workflow and does not replace those products.

Can I scrape private or deleted videos?

No. Do not bypass privacy or access controls. Deleted and private inputs should fail cleanly.

Does the comments endpoint return reply text?

No. It returns top-level comments. Reply counts may be present. Reply text needs comment-replies and a comment_id.

Can I send a profile URL to the videos endpoint?

No. Videos accept direct video URLs. Profiles and user-videos accept usernames.

Does search return users or sounds?

No. Current search is video-only.

Which key should I use for deduplication?

Prefer resource + id; fall back to a normalized canonical URL or username when the public ID is unavailable.