Upcoming changes to Tweets

Introduction

This document outlines the set of changes to Tweets so that people can express even more in 140 characters: 1) to allow for richer public conversations that are easier to follow on Twitter and 2) to ensure people can attach media to Tweets without sacrificing the characters they have to express themselves.

These changes touch many aspects of the Twitter platform. To that end, we have prepared these technical materials to help you transition your products and applications to the new Tweet format. The following sections will step through the planned technical changes.

Recent revisions:

  • Clarification on .@ syntax
  • Additional JSON sample payloads
  • Additional information added on the compose options (specifically the attachment_url, auto_populate_reply_metadata and exclude_reply_user_ids parameters)
  • Additional information on error codes associated with the new scenarios

Overview

The following chart and sample JSON demonstrate the differences between Tweet JSON objects in the various API endpoints (REST, streaming and Gnip), once the API changes described above are active. (If the formatting is difficult to read, please see the documentation and samples on our corresponding Github repo). Additional samples are available on Github and also via the Gnip support documentation.

Mode Objective Availability Details Example Tweets (JSON)
Compatibility Tweet payload will work with all existing integrations, regardless of the content of the Tweet Default for all REST APIs
  • text field is truncated to 140, as needed
  • truncated field is true
  • any entities only include those in the available 140 text range
Compatibility with additional extended_tweet in payload Maintain backwards compatibility with a non-breaking, payload addition Default for all Streaming and Gnip APIs
  • Includes new extended_tweet field in payload, containing:
    • full_text
    • display_text_range
    • entities
    • extended_entities
Extended Tweet payload contains all information to render Tweets that contain more than 140 characters

REST APIs only: add the below parameters to any endpoint:

  • tweet_mode=extended
  • full_text replaces text
  • truncated field is false
  • display_text_range delineates different sections of body for mentions/tweets/entities

What is changing?

We are simplifying the way that replies work on Twitter by moving some of the “scaffolding” of Tweets into display elements so that they no longer count towards the character limit within the Tweet.

  • Simplified replies: @names that auto-populate at the start of a reply Tweet will not count towards the character limit (but new non-reply Tweets starting with a @mention will count, as will @mentions added explicitly by the user in the body of the Tweet).
  • Media attachments: A URL at the end of Tweets generated from attaching photos, a video, GIF, poll, Quote Tweet, or DM deep link will also not count towards the character limit (URLs typed or pasted inside the Tweet will be counted towards the character limit as they do today).

This set of changes will introduce new limits around the numbers of specific elements that may be included as part of a Tweet (specifically, mentions).

These changes are shipping in stages. Our goal is to give developers and partners this advance notice of changes to the format of Tweets so that they can prepare their products and applications appropriately.

Compatibility, and what this means for developers

Backward and forward compatibility for third party clients and other API users are our primary considerations.

There are a number of areas that will be impacted by the change:

  • the public REST and Streaming APIs
  • the Ads API
  • the Gnip data products
  • display products, such as Twitter Kit for embedded Tweets and timelines displayed on iOS, Android, and Web.

Tweet object changes

The following things will change within Tweet payloads:

The displayed text in a Tweet does not exceed 140 characters, but - when usernames or attachment URLs are included at the appropriate points in the Tweet - the text content of the overall Tweet JSON object will be able to exceed 140 characters. Developers must avoid hard-coding length assumptions into their applications.

The text shall be logically divided into three regions:

A hidden prefix region that may contain one or more space-separated @mentions which shall not be rendered as part of the display text, but must instead be rendered as metadata.

A display text region, which remains a maximum of 140 characters in length.

A hidden suffix region that may contain one attachment URL which shall not be rendered as part of the display text, but must instead be rendered as metadata. This region is limited to containing a single URL entity that identifies an attachment resource: currently, one to four photos, a GIF, video, poll, Quote Tweet, or DM deep link.

  • Note: URLs for Quote Tweet or DM deep links that are typed or pasted into a Tweet will still count against the character limit. The new attachment_url parameter on the POST statuses/update endpoint will enable valid link formats to be attached to a Tweet. They will not count against the character limit when this method is used.

If the text contains a hidden prefix or suffix region, then the Tweet object shall contain values to identify the start and end indices of the region of the text to be displayed as the Tweet text.

Example payloads are provided in the appendix.

What does this look like?

This diagram shows the high-level change to Tweets, and the elements that will be hidden in the user interface.

image0

When rendered in apps or on the web, the hidden @mentions shall appear outside of the visible Tweet body, in a format similar to below. When a Tweet is in reply to multiple people, the name of the person whom the author is directly replying to should be prioritized.

image1

Terminology

Classic Tweet - A Tweet object where the total length of the text content does not exceed 140 characters. It may or may not contain leading and/or trailing text that shall be hidden by newer clients.

Extended Tweet - A Tweet object which includes hidden entities (e.g. leading @mentions and trailing attachment) and where the text content exceeds 140 characters in length. The display text region shall not exceed 140 characters.

Rendering modes

There will be two modes for rendering Tweet JSON objects to API clients: compatibility mode and extended mode. Compatibility mode is the default mode for the public REST and Streaming APIs and Gnip products, and is designed to not break existing clients.

REST API clients may opt into the extended mode via a request parameter.

In the future, an additional announcement will be made when the time is right to make a change for the rendering mode to default to extended mode.

Compatibility Mode JSON Rendering

In compatibility mode, Classic Tweets will be rendered exactly as today.

For Extended Tweets in compatibility mode, the following will be true:

  1. The existing text field will contain a truncated version of the Tweet text, followed by an ellipsis character, a space, and a shortened self-permalink URL. The total length of this text value shall not exceed 140 characters.
  2. The existing truncated field will be set to true.
  3. The existing entity fields (mentions, urls, media, etc.), will only contain entities that are fully contained within the text value. The from and to indices within each entity must be a valid code point index within the text value. The truncation point will avoid truncating mid-entity. A URL entity for the appended self-permalink will be appended to the list of entities.
  4. The payload may contain a new dictionary field named extended_tweet (this is specific to the Streaming and Gnip APIs). This will contain the following sub-fields:
    • full_text: contains the full, untruncated Tweet text.
    • display_text_range: an array of two unicode code point indices, identifying the inclusive start and exclusive end of the displayable content of the Tweet.
    • entities/extended_entities, etc.: The full set of entities.
  5. If the Tweet contains a Quote Tweet permalink URL, then the resulting embedded Quoted Tweet, if any, will still be included even if the permalink URL is not included in the truncated text.
  6. If the Tweet contains a URL entity that results in an attached card, then the card will still be included even if the original URL entity is not included in the truncated text.
  7. Since native media is only represented via entities, those will be missing from the truncated list of entities, but will be in extended_tweet.entities.

Extended Mode JSON Rendering

In extended mode, the following will be true both for Classic Tweets and Extended Tweets:

  1. The text field is no longer included; instead, the payload will contain a field named full_text, which contains the entire untruncated Tweet text.
  2. The payload shall contain a field named display_text_range, which is an array of two unicode code point indices, identifying the inclusive start and exclusive end of the displayable content of the tweet.
  3. The truncated field will be set to false.
  4. The entity fields will contain all entities, both hidden and displayable.

Limits

There will be restrictions placed on the content of the text. This is to improve the end-user experience, and to encourage high quality content. Tweets will be rejected at creation time if they exceed the new entity limits, via new API error codes. These restrictions will be enforced on all Tweets, regardless of overall character count (this represents a change to the existing methods that support creating new Tweets).

The numbers listed below are intended as initial guidelines.

  • overall Tweet text: limited to 3,000 Unicode code-points, after applying Unicode Normalization Form C.
  • @mentions: a limit of 50 @mentions per Tweet in the hidden region. This is enforced on the server side, so that users cannot exceed this number.
  • existing numbers and sizes of media attachments remain unchanged (up to 4 images represented by a single URL, or 1 GIF, or 1 video). Links that are added to the Tweet in order to link to media attached via the Twitter app or website (aka “native media links”) do not count, but links typed or pasted into the compose box may do so.

API changes

Public REST API endpoints

Composition

The REST API endpoints that create new Tweets (statuses/update) will accept a new boolean parameter when a Tweet is sent as a reply to a conversation: auto_populate_reply_metadata (true to enable, false to disable, false being the default). The existing in_reply_to_status_id must also be set. The leading @mentions will subsequently be looked up from the original Tweet, and added to the new Tweet from there. In cases where the original Tweet has been deleted, the reply will fail. This is a change to existing behaviour, where it has been possible to reply to a deleted Tweet ID.

For older clients that are not updated for the auto_populate_reply_metadata option, mentions will continue to be included in the body of the Tweet and the server will decide on how to render the new Tweet.

The auto_populate_reply_metadata option will append @mentions into the metadata as a reply chain grows, until the limit on @mentions is reached. In order to edit down the list of handles, an additional option, exclude_reply_user_ids, will enable specific IDs (apart from the leading one) to be excluded from a reply. This parameter is an optional, comma-separated list of user ids which will be removed from the server-generated @-mentions prefix.

Note that the leading @mention cannot be removed as it would break the in-reply-to-tweet-id semantics. Attempting to remove it will be silently ignored.

In order for a URL to not be counted in the body of the Tweet, a new attachment_url parameter will be available on statuses/update to allow a client to attach it to the Tweet without explicitly adding it to the Tweet text. This URL must be a Tweet permalink, or DM deep link. Arbitrary, non-Twitter URLs should remain in the Tweet text and will count against the 140 character limit. URLs passed to the attachment_url parameter not matching either a Tweet permalink or DM deep link will fail at Tweet creation and cause an exception.

Consumption

Any REST API endpoints that return Tweets will accept a new tweet_mode request parameter.

Valid request values are compat and extended, which give compatibility mode and extended mode, respectively.

The default mode (if no parameter is provided) is compatibility mode, to support older clients and display methods.

Tweets rendered in compatibility mode via the public REST API will not contain the extended_tweet field. REST API clients that wish to get the full text can instead opt into extended mode.

Error codes

Due to the limitations listed above, new API response and error codes will be introduced. These reflect the new content requirements listed in the Limits section above.

44 attachment_url parameter is invalid. Corresponds with HTTP 400. The URL value provided is not a URL that can be attached to this Tweet.
385 You attempted to reply to a tweet that is deleted or not visible to you. Corresponds with HTTP 403. A reply can only be sent with reference to an existing public Tweet.
386 The Tweet exceeds the number of allowed attachment types. Corresponds with HTTP 403. A Tweet is limited to a single attachment resource (media, Quote Tweet, etc.)

Public Streams

The Streaming API does not provide the same ability to provide query parameters to configure request options. Therefore, the Streaming API will render all Tweets in compatibility mode at this time.

Tweets rendered in compatibility mode for the streaming APIs, unlike for the REST APIs, will include the extended_tweet field for any extended tweet. This is necessary to avoid breaking existing clients by sending text that is longer than they expect in the existing text field, and also to provide the entirety of the data in a single stream. If there is an extended_tweet field, it will also include the ranges described above.

Streaming API consumers should update their code to first check for the presence of the extended_tweet dictionary, and use that in preference to the truncated data as is applicable for their use case. When extended_tweet is not present, they must fall back to using the existing fields.

In the future, a date for a switchover to extended mode will be announced, after which time apps should be able to process the newer Tweet payloads.

Gnip (REST and Streaming APIs)

In the case of Data products, both the REST and Streaming endpoints will follow a similar pattern to the public Streaming API, and the current versions of the data products APIs will render Tweets in compatibility mode, with the extended_tweet field.

The impact is intended to be a minimal, additive and opt-in, non-breaking change. Gnip customers will have to make a code change to “opt-in” to utilize the new additive fields when present. They may also want to prepare for the impacts of increased payload sizes, including storage and bandwidth implications.

In addition to payload changes, upon release of new Tweet payloads, Gnip operators and enrichments will begin to analyze the longer text and entities as opposed to the truncated version.

Tweet display on Web, iOS, Android

Web

Twitter’s web embed products are powered by our widgets.js JavaScript library which will be automatically updated to support new Tweet display formats without additional configuration needed from publishers.

iOS, Android

A future version of the Twitter Kit libraries will support retrieving extended Tweets and displaying the results inside new templates. An application developer will decide to update their version of Twitter Kit after it is available, build the new code with their app, and release the change as part of their regular app update process.

Tweet Web Intent, Twitter Kit Tweet Composer

We currently have no planned changes to the Tweet web intent, or to the Tweet Composer functionality included in Twitter Kit.