Short message index quick reference guide
This quick reference guide includes:
Search options
There are several options for searching the short message search index.
- The first is to enter keywords that automatically search across all metadata fields, including message-level and event-level text. Or, search specific field names and keywords. This method is the same as dtSearch.
- Another option is to search by query.
- The basic format for a query is: FIELD_NAME: Keyword, where FIELD_NAME is the name of a specific event-level metadata field. For example, timestamp.
- The table below lists four of the more common fields names you might use. For a complete list of searchable fields, see All searchable fields.
- The colon operator is similar to IS LIKE in that the query looks for any instances where the keyword appears in that field for an event.
- Keywords should correspond to the field type. For example, by searching the sender_display field, you might enter John Smith.
- Searching the timestamp field, you would enter a date-formatted keyword, such as 2024-02-09.
- You can also combine searches across different fields using OR and AND operators similar to traditional query languages. To view a list of search syntax, see Elastic Query Syntax.
Common fields
This section displays the four most common fields used for short message searching, along with tips for maximize your search results.
| Field name | Real name | Type | Description | Example |
|---|---|---|---|---|
| Sender Display | sender_display | Text | Display name of the sender. Display names are not standardized so they can be different in different platforms for the same people. | Lilliana Huff - P1 |
- If searching for multiple senders, make sure to
use parentheses and add AND, OR operators between the different objects. For example,
sender_display:(Dilan OR "John Smith" OR John*) - Sender display is not a standardized field. It is the user's display name in a given platform. Note: The display may be different across different platforms.
-
Putting quotation marks around the display name gives different results than searching without them. For example, searching
sender_display:"John Smith"searches for the exact phrase, whilesender_display:(John Smith)searches for John and Smith separately and returns both sets of results.
| Field name | Real name | Type | Description | Example |
|---|---|---|---|---|
| Timestamp | timestamp | Date | Returns the date and time the user sent the message. | 2021-01-05T09:15:45 |
- The format for date and time is
yyyy-mm-ddThh:mm:ss. When searching only a date, useyyyy-mm-dd. - When searching dates and times, use four digits for the year, and two digits for the month and day. For example,
2024-02-09. Using shortened date formats, such as 24-2-9 will not work. This same applies to the time format. You can use '00' for time as long as you complete the entire sequence: 00:00:00. - Use brackets [ ] to include dates and times on either side of the query. Use curly brackets { } to exclude dates and times on either side of the query.
- You can search for single dates or date ranges.
- If you want to find messages before a certain date, use
timestamp:< yyyy:mm:dd. For messages after a certain date, usetimestamp:> yyyy:mm:dd.
| Field name | Real name | Type | Description | Example |
|---|---|---|---|---|
| Message Body | message_body | Text | The actual text of the message. | I hear Sally has his ear. Tell Sally we'll take her to a nice dinner as a token of our appreciation in exchange for some business with Dr. Louis. |
- If you are only searching for text or want to search across all fields, you can enter the keyword directly in the search bar.
- If you are searching for multiple keywords, make sure to
use parentheses between the different objects. For example,
message_body: (Cat OR "litter box" OR dog*).
| Field name | Real name | Type | Description | Example |
|---|---|---|---|---|
| Event Type | event_type | Keyword | Multi-choice field. Options include message, disclaimer, join, or leave. | message |
If you are searching multiple event types, make sure to use parentheses between the different objects. For example, event_type: (join or leave).
Query examples
Use the following examples as a starting point for creating your query strings.
Example one
sender_display:(Abbie OR "Kyson Stanley") AND conversation_type:channel AND timestamp:[2012-10-13 TO 2023-12-12]
In English: Find messages sent by either Abbie or Kyson Stanley from October 13th 2012 and December 12th 2023, within a channel, not direct message.
Example two
sender_display: "Patience Mayo" AND message_body: "if you can cough" AND timestamp:[2012-10-13T09:15:00 TO 2023-10-13T15:15:00]
In English: Find all messages with 'Patience Mayo' in the display name, where the exact phrase "if you can cough" appears in the message body, and sent between 9:15 am on October 13th 2012 and 3:15pm on October 13th 2023.
Example three
is_edit:true OR deleted:true OR event_type:(join OR leave)
In English: Find all messages that were either edited or deleted, or a join or leave event.
Example four
message_body: (fraud OR "don't mention this" OR "take it offline") AND timestamp:<2023-10-01
In English: Find all messages containing the terms "fraud", "don't mention this", or "take it offline", sent before October 1, 2023.
Resources
Use the sections below to view a list of all searchable short message fields and a list of basic EQS operators and functions.
All searchable fields for short message searching
| Field name | Real name | Type | Explanation | Example |
|---|---|---|---|---|
| IDs | ||||
| Document ID | document_id | Keyword | Document ID of the RSMF File/Slice corresponding to certain messages | 2836477 |
| Conversations | ||||
| Conversation Type | conversation_type | Keyword | Multi-choice field. Can either be direct OR channel | channel |
| Conversation Platform | conversation_platform | Keyword | MS Teams | |
| Events | ||||
| Event Type | event_type | Keyword | Multi-choice field. Can either be message OR disclaimer OR join OR leave | message |
| Message Body | message_body | Text | Actual text of the message | I hear sally has his ear. Tell Sally we'll take her to a nice dinner as a token of our appreciation in exchange for some business with Dr. Louis. |
| Deleted | deleted | Boolean | States if someone deleted the message or not. True for deleted, False for not deleted. | false |
| Is edited | is_edit | Boolean | States if a user edited the message or not. If no one edited the message, the Message Body Previous field will be null. | false |
| Timestamp | timestamp | Date | States the date and time someone sent the message. | 2021-01-05T09:15:45 |
| Sender | ||||
| Sender Display | sender_display | Text | Display name of the sender. Note that the display names are not standardized and can be different in different platforms for the same people. | John Smith |
| Sender Email | sender_email | Text | john.smith@someemail.com | |
| Attachments & Reactions | ||||
| Attachment Displays | attachment_displays | keyword | these will contain file name so can search for file extensions but not 100% reliable because native type and file extension can be different in some situations. | |
| Attachment Count | attachment_count | Text | can check if there are no attachment or greater less than certain fresh hold. | |
| Reaction Types | reaction_types | Keyword | [":see_no_evil:",":shushing_face:",":football:"] | |
| Reaction Total Count Across Types |
reaction_total_count _across_types |
Integer | 5 | |
| Reaction Participants Display |
reaction_participants _display |
Text, Wildcard | ["Laura Jones","Michael Oliver"] | |
| Coding | ||||
| Responsive | custom_responsive | Boolean | This field reflects any message level coding decisions made on the ‘responsive’ field using the coding layout. |
Checking for any messages with coding – _exists_: custom_responsive Checking for any messages without coding – NOT _exists_: custom_responsive Checking for any messages coded as responsive – custom_responsive: true Checking for any messages coded as not responsive – custom_responsive: false |
| Privileged | custom_privileged | Boolean | This field reflects any message level coding decisions made on the ‘privileged’ field using the coding layout. |
Checking for any messages with coding – _exists_: custom_privileged Checking for any messages without coding – NOT _exists_: custom_privileged Checking for any messages coded as responsive – custom_privileged: true Checking for any messages coded as not responsive – custom_privileged: false |
| Notes | custom_notes | Text | This field reflects any message level coding notes made in the ‘notes’ field using the coding layout. |
Searched the same as any other text field Custom_notes: “search for any phrase” |
Elastic Query Syntax operators and functions
| Search function | Use | Query example |
|---|---|---|
| Basics |
In most cases, your search string consists the field name, followed by a colon, followed by the search criteria. Use quotes to search for an exact phrase. Omit field names to search all fields. Search results are not case sensitive. |
To search a specific field:
"Tom%Smith" matches Tom Smith in any field. |
| Brackets | Use square brackets for inclusive date ranges " [ ]" Use curly brackets for exclusive date ranges "{ }" |
Inclusive
[2023-01-01 TO 2023-21-31] result in dates between 2023-01-01 and 2023-12-31, including the start and end dates. Exclusive {2023-01-01 TO 2023-12-31} matches dates between 2023-01-01 and 2023-12-31 excluding the start and end dates. |
| Contains, IS LIKE | For CONTAINS, query the field, followed by a colon, followed by the query term. For IS LIKE, use the fuzzy operator. |
|
| Date ranges |
Use TO when specifying date ranges. |
{* TO 2023-01-01} matches all dates occurring before 2023-01-01 { 2023-01-01 TO *} matches all dates occurring after 2023-01-01 [2023-01-01 TO 2023-12-31] matches all dates in 2023 |
| Escape characters | Use the escape character "\" to force a literal interpretation of special characters and system characters. | Company\'s matches Company's |
| Exact phrase |
You must use quotes to search for an exact phrase. | "Tom Smith" matches Tom Smith |
| File size | Use to further narrow attachment results | attachment_max_size:>=20 returns documents with attachments that are greater than or equal to 20. The default unit of measurement is bytes. |
| Fuzziness | Use the ~ operator for fuzzy searches. | fokl~ matches both folk and folks |
| Grouping sub-queries | Use parenthesis to group queries and sub-queries. | ("Tom" AND ("Smith" OR "Jones")) returns Tom Smith and Tom Jones |
| Operators | Use AND, OR, NOT | "Bob" OR "Cat" returns documents containing the words Bob or Cat. |
| Punctuation |
ElasticSearch treats most punctuation and symbols as word breaks. Use the escape character "\" to force a literal interpretation of special characters. |
Company's matches Company s Tom_Smith matches Tom Smith Tom%Smith matches Tom Smith Company\'s matches Company's Company\'s matches Company's "\\:wave\\:" returns the wave emoji |
| Proximity | ("Term 1 Term 2" ~N) where N is the distance between terms. | ("quick fox" ~5) matches quick brown fox |
| Spaces | Use the percentage character to include a space. | "Tom%Smith" matches Tom Smith |
| Wildcards | Use ? for a single character. User * for multiple characters. ElasticSearch ignores wildcards placed inside of quotes. Cannot use wildcards to search for an entire phrase. |
"Sales agreement"* matches Sales agreement, Sales agreements "Sales agreement*" matches Sales agreement |
| Emojis | Use the emoji or use quotation marks around the text version of an emoji. | ❤ or ":heart:" |
