Insights Reports and Signal Data Requirements

Although there are many fields that are relevant to signals, this topic describes which fields are required for each type of signal. Without these fields, reports and dashboards within App Insights may not function as expected.

Signal Types

There are five types of signals:

• Annotation. Annotation signals are generated when a user bookmarks, likes, or comments on a document. Annotation signals are likewise generated when the user removes a bookmark, like, or comment.

  Annotation signals are generated by App Studio. If you are not using App Studio, this type of signal is not relevant to your search application.

• Click. Click signals are generated when a user clicks on a page element that is being monitored by the search app. Click signals are sent from the search app to Fusion.

• Login. Login signals record information about specific users when they log in to an application. This includes a time stamp and various session details.

• Request. A request signal is generated by a front-end search app and captures the raw user query and other contextual information about a user and their journey through the search app.

• Response. Response signals are automatically generated by a query pipeline when the signals feature is enabled for a collection.

  Important Because response signals and their fields are automatically generated, this topic does not cover what response signal fields are required.

Required Fields

Signals

The following table describes which fields are required for annotation, click, login, and request signals.

Requests (or queries) can also require additional, available user data for the search.

Field	Type	Description	Example	Required
id	string	Unique ID for the signal.	b0ee5307-6223-4150-ac5a-d0d8113aa480	✅ Annotation ✅ Click ✅ Login ✅ Request
doc_id	string	Product ID or Item ID of the clicked result.	NMDDV	✘ Annotation ✅ Click ✘ Login ✘ Request
date	timestamp	Timestamp of when the signal was generated. This timestamp follows Unix epoch time formatting.	1518717749409	✅ Annotation ✅ Click ✅ Login ✅ Request
user_id	string	Unique ID for the user that generated the signal.	admin	✅ Annotation ✅ Click ✅ Login ✅ Request
session	string	Unique ID for the user’s browser session.	ef4e00cd-91bb-45b4-be80-e81f9f9c5b27	✅ Annotation ✅ Click ✅ Login ✅ Request
host	string	Host name of the server which is hosting the app that is generating the signal.	x.y.z	✅ Annotation ✅ Click ✅ Login ✅ Request
ip_address	string	IP address of the user that generated the signal.	80.6.99.35	✅ Annotation ✅ Click ✅ Login ✅ Request
app_id	string	Name of the application that is generating the signal.	myApp	✅ Annotation ✅ Click ✅ Login ✅ Request
annotation_type	string	Type of the annotation signal, either “comment”, “bookmark”, or “like”. Required fields for “comment”: • comment. The comment itself. • target. The target that the comment refers to. Required fields for “bookmark”: • title. The bookmark title. • url. The bookmark url. Required fields for “like”: • -• -	bookmark	✅ Annotation ✘ Click ✘ Login ✘ Request
query	string	Terms of the query.	ipad	✘ Annotation ✅ Click ✘ Login ✅ Request
fusion_query_id	string	Unique ID for the query that is automatically generated from the Fusion response signal.	ABkaEA11	✘ Annotation ✅ Click ✘ Login ✘ Request
filter	array of string	List of filters associated with the query, which in turn is associated with signal.	[“type/tablet”,”category/electronics”]	✘ Annotation ✅ Click ✘ Login ✅ Request
ctype	string	Type of click.	result	✘ Annotation ✅ Click ✘ Login ✘ Request
res_pos	number	Position of the clicked result within the list of results.	3	✘ Annotation ✅ Click ✘ Login ✘ Request
res_offset	number	Result page.	2	✘ Annotation ✅ Click ✘ Login ✘ Request
url	string	URL of the page that the signal originated from.	http://localhost:8080/products/search	✘ Annotation ✘ Click ✘ Login ✅ Request
path	string	URL path of the page that the signal originated from.	/search	✘ Annotation ✘ Click ✘ Login ✅ Request
page_title	string	Title of the page that the signal originated from.	Search Page	✘ Annotation ✘ Click ✘ Login ✅ Request

Insights Reports

The table below describes which fields are required to generate the various reports within App Insights:

Report	Signal Types Used	Required Field	Type	Description	Example
Search queries	✘ Annotation ✅ Click ✘ Login ✅ Request ✅ Response	query	string	Terms of the query.	physics
Facets used	✘ Annotation ✘ Click ✘ Login ✅ Request ✘ Response	filter_field	array of string	List of filters associated with the query, which in turn is associated with signal.	["categories_s"]
Facet filters applied	✘ Annotation ✘ Click ✘ Login ✅ Request ✘ Response	filter	array of string	List of applied filters associated with the query, which in turn is associated with signal.	["categories_s/math.OA"]
Application servers	✘ Annotation ✅ Click ✘ Login ✅ Request ✘ Response	host	string	Host name of the server that is hosting the app, which in turn is generating the signal.	lucidworks
Response times	✘ Annotation ✘ Click ✘ Login ✘ Request ✅ Response	time	number	Response time (in milliseconds).	17
URLs clicked	✘ Annotation ✅ Click ✘ Login ✘ Request ✘ Response	url	string	URL of the page that was selected from the search results. This typically results from a click signal.	http://arxiv.org/pdf/astro-ph/0611688v1
Search pages	✘ Annotation ✘ Click ✘ Login ✅ Request ✘ Response	page_title	string	Title of the page that the signal originated from.	Search Preview
Search platforms	✘ Annotation ✅ Click ✘ Login ✅ Request ✅ Response	platform	string	Name of the search platform.	fusion
Visitor countries	✘ Annotation ✅ Click ✘ Login ✅ Request ✘ Response	geo_country_name	string	Originating country of the user.	United States
Visitor cities	✘ Annotation ✅ Click ✘ Login ✅ Request ✘ Response	geo_city_name	string	Originating city of the user.	San Francisco
Browsers	✘ Annotation ✅ Click ✘ Login ✅ Request ✘ Response	browser_name	string	Name of the browser that generated the signal.	Firefox
Operating System	✘ Annotation ✅ Click ✘ Login ✅ Request ✘ Response	os_name	string	Name of the operating system used by the user.	Ubuntu
Device types	✘ Annotation ✅ Click ✘ Login ✅ Request ✘ Response	os_device	string	Type of the device used by the user.	Computer
Websites people are coming from	✘ Annotation ✅ Click ✘ Login ✘ Request ✘ Response	referrer_domain	string	URL of the page that the signal originated from.	http://www.google.com
Users	✘ Annotation ✅ Click ✘ Login ✅ Request ✘ Response	user_id	string	Unique ID for the user that generated the signal.	admin
User domains	N/A	domain	string	Domain of the user that generated the signal.	lucidworks.com
Types of event	✅ Annotation ✅ Click ✅ Login ✅ Request ✅ Response	type	string	Type of the signal.	click

Head Tail analysis uses different fields depending on signal type. This report is populated by running a head tail job in Lucidworks Search. With a collection called foo, for example, run the foo_head_tail job.

Signal Types Used	Required Field	Type
✘ Annotation
✅ Click	type	string
✅ Click	query	string
✘ Login
✘ Request
✅ Response	count_i	number

Fusion jobs

The following tables specify required fields for specific Fusion jobs:

Job: click_signal_aggr

Field	Required
user_id	no
query	yes
doc_id	yes
count_i	yes
fusion_query_id	depends
type	yes
session_id	optional
id	N/A
res_offset	optional
res_pos	optional
filters	optional
type = response	depends
dependency	N/A

Job: phrase_extraction

Field	Required
user_id	no
query	yes
doc_id	no
count_i	yes
fusion_query_id	no
type	yes
session_id	N/A
id	N/A
res_offset	N/A
res_pos	N/A
filters	N/A
type = response	N/A
dependency	N/A

Job: spell_correction

Field	Required
user_id	no
query	yes
doc_id	no
count_i	yes
fusion_query_id	no
type	yes
session_id	N/A
id	N/A
res_offset	N/A
res_pos	N/A
filters	N/A
type = response	N/A
dependency	N/A

Job: synonym_detection

Field	Required
user_id	no
query	yes
doc_id	yes
count_i	yes
fusion_query_id	no
type	yes
session_id	N/A
id	N/A
res_offset	N/A
res_pos	N/A
filters	N/A
type = response	N/A
dependency	phrase, misspelling

Job: head_tail

Field	Required
user_id	no
query	yes
doc_id	yes
count_i	yes
fusion_query_id	no
type	yes
session_id	N/A
id	N/A
res_offset	N/A
res_pos	N/A
filters	N/A
type = response	N/A
dependency	N/A

Job: _user_item_prefs_agg

Field	Required
user_id	yes
query	no
doc_id	yes
count_i	yes
fusion_query_id	no
type	yes
session_id	yes
id	yes
res_offset	yes
res_pos	yes
filters	yes
type = response	yes
dependency	N/A