Description: C:\Users\yergens\Dropbox\Shared Folders\DeanerKevin\synthesis-logo\jpeg\LogoColorTextRight.jpeg

 

 

Synthesis Version 2.3

User Manual

 

Dean Yergens

Kevin Mackie

 

 

 

Synthesis Version 2.3

User Manual

Release 2.3 (October 2013)

 

 

 

 

 

 

 

Description: C:\Users\yergens\Dropbox\Shared Folders\DeanerKevin\synthesis-logo\jpeg\LogoColorTextRight.jpeg


 

CONTENTS

introduction... 7

Synthesis Best Practices.. 8

Synthesis ten steps.. 9

Synthesis Version 2.3 Release... 10

Synthesis Version 2.3 tutorial.. 12

Creating a Synthesis Project. 12

Importing references from PubMed.. 14

The Synthesis Application.. 18

Working with Settings. 20

Downloading Articles (PDFs). 23

Synthesis Tabs - Paper. 25

Synthesis Tabs - Stats. 27

Synthesis Tabs - Bibliography. 29

Synthesis Tabs - BibTeX. 30

Synthesis Tabs - Duplicates. 31

Synthesis Tabs - Word Cloud.. 32

Inclusion/Exclusion Criteria.. 33

Export to CSV.. 35

Export to RIS (Reference Manager format). 38

Search for Missing Paper (Google Scholar Search). 39

Import Missing Article (PDF). 41

Launch External PDF Viewer. 42

Import Highlighted Text and Comments from PDF. 44

Secondary Reviewer. 45

Contrast Primary and Secondary Reviewers. 47

Toggle On or Off the PDF Viewer. 49

Export Synthesis Project. 51

Import Synthesis Project. 52

Synthesis Tools - Kappa Calculator. 56

Synthesis Tools - Sample Column.. 58

Synthesis Tools - Grey Literature. 59

Synthesis Tools - Scan for PDFs. 61

Synthesis Tools - Extract PDF Images. 62

Preferences - ezProxy. 63

Register Synthesis Software (User License). 64

importing and de-duplicating references    66

Verification of Duplicates. 68

importing and de-duplicating references - Case Study    71

Supported Source Databases.. 74

Synthesis Version 2.3 Import Validation Check. 74

Importing from PubMed.. 75

Importing from Ovid Medline/Embase/Etc. 76

Importing from Web of Knowledge. 78

Importing from IEEE. 80

Importing Scopus (Elsevier). 81

Importing ProQuest. 83

Importing EndNote. 85

Importing Grey Literature (BibTex). 86

searching in Synthesis.. 87

external applications for Synthesis.. 89

Export to RefWorks. 90

Export to EndNote/Reference Manager. 92

Using BibTeX in Latex/Sweave. 95

Using Synthesis with Carrot2 (Clustering Engine). 98

Using Synthesis with HSI WebExcel. 101

Annotate Visible Columns. 102

Synthesis Project File Folder... 104

Working with Synthesis Flowchart.. 105

Backing up Synthesis Projects.. 106

APPENDIX a - KEYWORDS.. 111

 

introduction

Synthesis is an application for conducting scoping and systematic literature reviews.  Synthesis sets out a formal workflow process for conducting these reviews and tries to automate tasks that are repetitive and labor intensive. 

Description: C:\Users\yergens\Dropbox\Yergens\Photos\Fotolia\Fotolia_28844937_Subscription_Monthly_XL.jpg

Figure 1 - Students doing Systematic Reviews the old labor intensive way

We saw was many graduate students conducting systematic reviews as part of their research in the fields of Epidemiology, Health Services Research and Population health and using an array of software applications in a non-unified manner.  Even though a Systematic Review is to have structure and a formal methodology, there did not exist the software tools to streamline the process in the most efficient manner resulting in students performing a lot of manual tasks that we thought could be automated in order to improve the workflow.

We searched the Internet to try to find software focused on Scoping and Systematic reviews and were unable to find any software that fitted our requirements and vision for how this process should occur. 

Synthesis Best Practices

 

·       Use the Synthesis Project Flowchart

·       Make Backups: especially before Importing project files

·       Document your project (including importing data) in Settings.

·       Use the Abstract and Paper keyword ability for automatically highlighting text (located in Settings)

·       Learn how to use the Search function.  It’s extremely powerful when searching PDF documents.


 

Synthesis ten steps

 

The following is a 10 step program suggestion for working with Synthesis.

1.     Create New Project

2.     Import Citations

3.     Review Duplicates

a.     Project > Project Settings: Select Duplicate Mode

b.     Sort by Title: Review titles

c.     Include/Exclude for Duplicate

4.     Configure Settings 

a.     Project > Project Settings

b.     Add Title and Description

c.     Type in Keywords you want highlighted

                                              i.     Add Exclude Reasons

5.     Review References (inclusion/exclusion)

a.     Hint: use Search Feature to filter meaningful or non-meaningful references

6.     Download All Papers

7.     Create Custom Columns

a.     Annotate Visible Rows

8.     Export to CSV

a.     Import into Excel, STATA, etc

9.     Export to RIS

a.     Import into RefWorks, EndNote, etc

10.  Cite Synthesis in your Paper

 

Synthesis Version 2.3 Release

Major Changes:

·       Addition of Word Cloud for navigating search results.

·       Ability to extract all images/figures from PDFs

·       Addition of ability to add Grey Literature

·       Ability to Toggle On or Off the PDF Viewer

·       Ability to import previously imported PDFs from other projects.

·       Importing and de-duplicating of references is now handled slightly different than previous versions.  This change was implemented to deal with two issues.  The first was to improve the way duplicates were reported to the user when dealing with multiple large reference files being imported.  The second issue was to improve the updating of the literature searches. 

·       Saves all the Source Database import files in your project folder under /import.

 

Minor Changes:

·       Export to EndNote fixed.  EndNote requires that when importing references (in the NLM PubMed format which Synthesis uses) have each field descriptor use a specified number of character spaces.  This means that "TI - " and "TI    - " are handled differently.

·       PubMed import fixed.  Sometime in August 2012 PubMed changed their file format when exporting.  The change in their format meant that there may not be a title if the reference was a foreign paper as the title for foreign languages is stored in another field. 

 

Other:

·       An updated Synthesis Summary Sheet has been developed.  It is now two pages and includes new information.

·       You can Include or Exclude references by pressing CTRL-I (In) or CTRL-O (out).

·       For Mac users, using the Command key for copying and pasting does not work.  Use the CRTL key instead (CTRL-C cut and CTRL-V paste).  This is because Synthesis is a cross platform application (Windows, Mac and Linux).

Synthesis Version 2.3 tutorial

Creating a Synthesis Project

When you start Synthesis for the first time you will get a dialogue box (Figure 2 - Synthesis First Start Up Screen) that says “Open Project”.  Because you are starting Synthesis for the first time there will be no projects listed. 

Description: Screen shot 2012-10-01 at 3.22.20 PM.png

Figure 2 - Synthesis First Start Up Screen

To create a new project, click on the “New Project” button and a new dialogue box (Figure 3 - New Project Dialog Box) will open.

We will call this project “NaturalLanguageProcessing” and then click on the OK button.

When a new project is created a folder will be created in your user directory under a folder called “LiteratureReview”.  This folder will store the Synthesis project database and also all the PDF documents downloaded.

For Windows users the location will be under:

C:\users\YOUR_NAME\LiteratureReview\NaturalLanguageProcessing

For Mac and Linux users the location will be under:

\YOUR_NAME\LiteratureReview\NaturalLanguageProcessing

Note: The LitertaureReview folder will only be created once you create a new project (not after you install Synthesis).

 

Description: Screen shot 2012-10-01 at 3.22.50 PM.png

Figure 3 - New Project Dialog Box

The next step will be to import some references into the Synthesis project.


 

Importing references from PubMed

 

To get some references for our Synthesisproject, we will go to the http://pubmed.org website.  We will type in the query (as seem in Figure 4 - PubMed.org Main Screen):

 “natural language processing” OR “nlp”

 

Note: Natural Language Processing is a computer science approach for helping in information retrieval from textual information.

 

Figure 4 - PubMed.org Main Screen

To get the reference file, we need to:

·       Click on “Send To” in the upper right side of the window.

·       Select “File” under “Choose Destination”

·       Select “Medline” under “Format”

·       Click the “Create File” button.

 

This will then automatically download a file called “pubmed_results.txt”.

We now return to the Synthesis application and click the “Import Abstracts” button.  This will then open a new dialog window (Figure 5 - Import Abstracts Open File Dialog Window) for selecting the abstracts PubMed file.  Once selecting the “pubmed_results.txt” file click the “Open” button which will then import the abstracts into Synthesis.

Description: Screen shot 2012-10-01 at 3.25.26 PM.png

Figure 5 - Import Abstracts Open File Dialog Window

While the abstracts are being imported a “Importing Abstracts” dialog window (Figure 6 - Importing Abstracts Dialog Window) will appear displaying the status of the importation.

 

 

Description: Screen shot 2012-10-01 at 3.25.44 PM.png

Figure 6 - Importing Abstracts Dialog Window

Once the abstracts have been imported as seen in “Figure 6 - Importing Abstracts Dialog Window” some descriptive statistics of the import are shown.  These statistics are as follows:

 

·       Abstracts Imported: This is the total number of Abstracts processed.  This number will match the downloaded abstracts number you received from the source database (i.e. PubMed).

·       Precise match (skipped): These abstracts are not imported.  These are the abstracts that are duplicates within the same source database.  For example, if you imported from PubMed in May 2012 and then again in September 2012 using the same search query, Synthesis would not import (or count as a duplicate) any abstracts already in the project database.  This allows the researcher to “refresh” their literature search.

·       Duplicate Abstracts Imported:  This is the number of duplicate abstracts found.  This happens when you have the same abstract from different database sources (i.e. PubMed and Embase).

 

 

Description: Screen shot 2012-10-01 at 3.26.21 PM.png

Figure 7 - Synthesis Imported Abstracts

 

When you click on the “OK” button in the “Figure 6 - Importing Abstracts Dialog Window” you will now return to the Synthesis main application (Figure 7 - Synthesis Imported Abstracts) and all your imported abstracts will be shown.

 

If you click the Stats tab (Figure 7 - Synthesis Imported Abstracts) you will get the descriptive statistics on the references.  More on this in Section: Synthesis Tabs - Stats.

 

Note for Future Reference: Synthesis stats will only calculate the references being displayed in Synthesis main application window (this is dependent on the Search filter and the Primary and Secondary Reviewer Abstracts and Full Text drop downs). 

 

 


 

The Synthesis Application

Description: Screen shot 2012-10-01 at 3.27.23 PM.png

 

 

 

The Synthesis application is composed of three main components.  They are as follows:

 

·       Reference List – the reference list is located in the top left section of the Synthesis application.  The Reference list contains information about the primary and secondary abstract and full text inclusion and exclusion classification; individual references (Year, Author, Title, Journal and Issue) and any custom columns the researcher may have added. 

·       Tab component – the tab component is located in the bottom left section of the Synthesis application.  The Tab component contains several tabs identified as Paper, Stats, Bibliography, BibTex and Duplicates.

·       PDF Viewer - the PDF viewer is located on the right side of the Synthesis application.  It can be turned on or off in the Settings.

 

All of the various components can be resized by clicking, holding and dragging the separator between the various components in the Synthesis application.

 

Synthesis functions by classifying references into three categories: “Unknown”, “Included” and “Excluded” for abstracts and full-text.  Both the primary and secondary researcher can classify references this way.  

The researcher can see what references are classified under each category by choosing form the Abstract or Full Text drop down menu found in the “Primary” or “Secondary” Researching rating in the Reference List component.

 

At the top of the Synthesis application is a Search field that allows a researcher to search for keywords both in the abstract or full text section.  This is an extremely powerful feature of Synthesis and is explained in detail in the section “Searching in Synthesis”.

 

The Tabs will only display the references being shown in the reference list section.  This is affected by the search query and the primary and secondary abstract and full text inclusion/exclusion pull-down menus.  You can see the number of references being shown right under the reference list section as “displaying X references”. 

 


 

Working with Settings

 

Synthesis has the ability to configure some settings in the program that may help with improving the workflow and data management.  The key features of the settings include:

·       Setting the Primary and Secondary reviewer status.

·       Highlighting words in the Abstracts and PDF for the easy identification of keywords.

·       Adding new Exclusion categories for Abstracts and Full Text for helping segment exclusions.

·       Adding custom data collection columns for helping in the data extraction phase.

 

To access the Settings (Figure 8 - Synthesis Settings Window), you need to:

 Select Project > Settings

Description: Screen shot 2012-10-01 at 3.31.47 PM.png

Figure 8 - Synthesis Settings Window

 


 

The following describes the various settings options:

·       Display Primary Researcher’s Rating:  This checkbox will turn on or off the primary researcher’s fields (Abstract and Full Text include/exclude drop down field and the Comments field in the abstract) on the main Synthesis screen.

·       Display Secondary Researcher’s Rating:  This checkbox will turn on or off the secondary researcher’s fields (Abstract and Full Text include/exclude drop down field and the Comments field in the abstract) on the main Synthesis screen.

·       Project Title:  This is where the researcher would enter the project’s title.

·       Project Description:  This is where the researcher would enter the project’s description.  For best practices we suggest including the literature search strategy and the dates that the source database references were imported in Synthesis.  Maintaining good records is good practice.

·       Abstract Keywords:  This section allows the researcher to enter words that they would like highlighted in the abstract section of the main Synthesis window.  The format of entering keywords is one keyword per line and in the case of multiple words to put quotations around the phrase (i.e. “logistic regression”).

·       Full Text Keywords:  This section allows the researcher to enter words that they would like highlighted in the full text section of the main Synthesis window.  The format of entering keywords is one keyword per line and in the case of multiple words to put quotations around the phrase (i.e. “linear regression”).

·       Abstract Exclude Reasons:  This section allows the researcher to include custom exclusion reasons in the pull down fields for the abstract review.  For best practices we add the pre-word “Exclusions - ”  to all of our exclusions reasons for easy identification in the Exclusion pull-down menu.  We also include the exclusion “Exclusions – Read Later” as we sometimes find we don’t have the attention span at that point of time to categorize the reference.   Note: When working out the kappa score – all exclusion reasons are grouped as one for the analysis.

·       Full Text Reasons:  This section allows the researcher to include custom exclusion reasons in the pull down fields for the full text review.  Note: Read “Abstract Exclude Reasons” for best practices.   

·       Custom Columns:  This section allows the researcher to add custom columns to the spreadsheet-like interface on the Synthesis main window to allow for data entry.  Some examples of this may be recording the Country of the study, Study Design and statistical methods used in the analysis.  

·       PDF Viewer:  This checkbox will turn on or off the PDF viewer on the main Synthesis window.  This improves the speed of doing certain tasks with Synthesis such as navigating a large number of references.

 

The following (Figure 9 - Abstract Keyword Highlighting) shows the abstract keywords being automatically highlighted as identified in the Abstract Keywords Setting.

 

Description: Screen shot 2012-10-01 at 3.34.49 PM.png

Figure 9 - Abstract Keyword Highlighting

 

 


 

Downloading Articles (PDFs)

There are several ways that Synthesis can import PDFs into the Synthesis project Database.  These are as follows:

 

·       Download the Individual PDF – Downloads the individual paper regardless of whether the reference is marked as Unknown, Included or Excluded.

·       Download All Papers – Downloads all papers that have been marked as “Included”.

·       Import from File – Allows the Researcher to import a PDF file from their computer.

 

Download the Individual PDF – When the researcher clicks the “Download” button on the Abstract component of the Synthesis main window, Synthesis will attempt to download the individual paper regardless of whether the reference is marked as Unknown, Included or Excluded.

 

Download All Papers: When you click on the “Download all Papers” button in the top section of the Synthesis Main window, Synthesis will then start to automatically download all papers that have been marked as “Included” in the primary or secondary abstracts inclusion/exclusion menu.

 

Import from File – When the researcher clicks the “Import from File” button on the Abstract component of the Synthesis main window a Open File dialog window will appear and the researcher can select the PDF to import.  This allows the Researcher to import a PDF file from their computer.  Note: This option is most frequently used with the “Search Google” button as a way of identifying the article through Google Scholar.

 

Note: Synthesis does have a ezProxy server built into the software, so a researcher can attached directly to the Universities network if they want to retrieve PDFs that their university has access to.  Otherwise, Synthesis is only able to download PDFs that the public has access to.  Note: We recommend that a user use the institutions VPN if available instead of the ezProxy server.

 

Note: Synthesis may not be able to retrieve all PDFs even if the University library has access to them and/or there is a DOI associated with the reference.  There may be several reasons for this – such as the PDF being embedded several layers into the Journal’s webpage.

 

When a PDF is downloaded or imported into Synthesis the reference is marked as “Yes” in the “Have PDF” column and the PDF is displayed in the PDF viewer as seen in “Figure 10 - Downloading PDFs”. 

 

 

Description: Screen shot 2012-10-01 at 3.35.44 PM.png

Figure 10 - Downloading PDFs

When PDFs are downloaded or importing in the Synthesis software, Synthesis saves all the PDFs in the folder:

C:\Users\YOURNAME\LiteratureReview\PROJECT

 

Synthesis also changes the name of the PDF to the following format:  YEAR_FIRSTAUTHOR_TITLE.pdf as seen in the following example:

 

2012_Bjorne_J_University_of_Turk_in_the_BioNLP_11_Shared_Task.pdf


 

Synthesis Tabs - Paper

Description: Screen shot 2012-10-01 at 3.36.03 PM.png

Figure 11 - Paper Tab

 

When you click on the Paper tab, Synthesis will display abstract information in the lower left hand window.  If a PDF has already been associated with this abstract it will be displayed in the right hand window.

 

The follow information is displayed about the abstract:

 

·       Title: The title of the abstract.

·       Journal:  The journal and issue of the abstract.

·       Authors:  The authors of the abstract.

·       PDF: The DOI of the abstract (web-location of the PDF)

·       Source: The source database that this abstract came from.  If the abstract was imported from several source databases then all of those source databases will be shown.

·       Abstract: The abstract of the reference.

·       Mesh Headings: The Mesh Headings (or keywords depending on the source database) of the abstract.

 

Note: You can also drag and drop a reference from Synthesis into any word processing program (i.e. Microsoft Word, Text Edit, etc) and the reference will appear in the format:  “Author(s). Title.Journal.Issue.” As seen below:

Brown MP, Grundy WN, Lin D, Cristianini N, Sugnet CW, Furey TS, Ares M Jr, Haussler D. Knowledge-based analysis of microarray gene expression data by using support vector machines.ProcNatlAcadSci U S A. 2000 Jan 4;97(1):262-7.

 

 


 

Synthesis Tabs - Stats

 

Description: Screen shot 2012-10-01 at 3.36.24 PM.png

Figure 12 - Stats Tab

 

When you click on the Stats tab, Synthesis will display some summary descriptive statistics about the references currently being displayed in the Synthesis main window. 

 

The Descriptive Statistics are:

·       Abstract Kappa: This is the kappa score calculated between the Primary and Secondary reviewer for the Abstracts.  This kappa score is only calculated when 100% of the Abstracts have been categorized as either “Included” or “Excluded”.

·       Full Text Kappa: This is the kappa score calculated between the Primary and Secondary reviewer for the Full Text articles.  This kappa score is only calculated when 100% of the Full Text articles have been categorized as either “Included” or “Excluded”.

·       Basic Stats – Unique Papers: This is the total number of abstracts that have been imported into the Synthesis project.

·       Basic Stats – Excluded Duplicates:  This is the number of unique abstracts in the Synthesis project.

·       Basic Stats – Papers with a DOI:  This is the number of abstracts that have a direct reference to the PDF.  Some journals or databases have relatively few direct links. 

·       Source Database (includes duplicate papers):  This section lists the source databases such as PubMed or Embase and the number of abstracts imported from it.  This section includes duplicate papers.

·       Source Database (includes duplicate papers):  This section lists the source databases such as PubMed or Embase and the number of abstracts imported from it.  This section does not include duplicate papers.

·       Top Journals:  This section lists the top 20 journals in the Synthesis project.

·       Top Mesh Headings:  This section lists the top 20 mesh heading (or keywords for some source databases) in the Synthesis project.

·       Publication Year:  This is a breakdown by publication year of the abstracts.

 


 

Synthesis Tabs - Bibliography

 

Description: Screen shot 2012-10-01 at 3.36.40 PM.png

Figure 13 - Bibliography Tab

 

When you click on the Bibliography tab, Synthesis will display abstract information in the lower left hand window.

 

The format of this information is Reference (Author, Title, Journal, Issue), Abstract, Mesh and Comments.

 

Reference: Lee TY, Chang WC, Hsu JB, Chang TH, Shien DM. GPMiner: an integrated system for mining combinatorial cis-regulatory elements in mammalian gene group. BMC Genomics. 2012;13Suppl 1:S3. Epub 2012 Jan 17.
Abstract: BACKGROUND: Sequence features in promoter regions are involved in regulating gene transcription initiation. Although numerous computational methods have been developed for predicting transcriptional start sites (TSSs) or transcription factor (TF) binding sites (TFBSs), they lack annotations for do not consider some important regulatory features such as CpG islands, tandem repeats, the TATA box, [Synthesis Note: the rest of the abstract removed]

MESH: Algorithms, Animals, Base Composition/genetics, Binding Sites/genetics, Computational Biology/*methods, CpG Islands/genetics, Databases, [Synthesis Note: the rest of the abstract removed]
Primary Comment: This paper is a good example of the GPMiner application.

 

Note: The Bibliography tab is the only way to export the Comments section for the primary and secondary reviewer.

Synthesis Tabs - BibTeX

 

Description: Screen shot 2012-10-01 at 3.36.54 PM.png

Figure 14 - BibTeX Tab

 

When you click on the BibTeXtab, Synthesis will display abstractBibTex information in the lower left hand window.  BibTex is a format mainly used in with Latex in the fields of Engineering, Computer Science, Statistics and other sciences for managing bibliography information.

 

An example of a BibTex reference is as follows:

 

@article{lee2012,

author = {Lee TY,ChangWC,HsuJB,ChangTH,Shien DM},

title = {GPMiner: an integrated system for mining combinatorial cis-regulatory elements in mammalian gene group.},

journal = {BMC Genomics. 2012;13 Suppl 1:S3. Epub 2012 Jan 17.},

year = {2012},

}

 


 

Synthesis Tabs - Duplicates

 

Description: Screen shot 2012-10-01 at 3.36.40 PM.png

Figure 15 - Duplicates Tab

 

When you click on the Duplicates tab, Synthesis will display duplicate abstract information in the lower left hand window.

 

Synthesis will show the primary reference and then the duplicate reference.  In many cases it is very small differences in identifying a duplicate.  Some examples are:

·       One source database using symbols “<” compared to another source database having typed “less than”

·       One source database having included copyright information in the abstract compared to another source database.

 

Note: A future release of Synthesis (scheduled for Synthesis.3) will treat duplicate references as a new category “Duplicate” along with “Unknown”, “Included” and “Excluded”.  This will allow the researcher to move a “Duplicate” paper into the “Unknown” category in case the Synthesis de-duplicating algorithm incorrectly indentifies an abstract as a duplicate.

 

 

 

Synthesis Tabs - Word Cloud

 

 

When you click the Word Cloud tab, Synthesis will display a word cloud.  A word cloud displays the most frequent words from a text document and increases the font size for the words that are more frequent.  Commonly used words such as "the", "and", "or", etc are removed from the word cloud. 

 

The word cloud is generated based upon the most frequent words in the title or in the abstract.  You can select which one, by clicking the radio buttons.

 

There is also a slider that allows you to increase or decrease the number of words being displayed. 

 

If you click on a word, then that word is added to the search filter which will allow you to drill down on the documents.  As each word is clicked upon it is appended using the "AND" symbol.

 

i.e. title:Canada AND abstract:health

 

 


 

Inclusion/Exclusion Criteria

Working with the Inclusion and Exclusion classification is an important feature of Synthesis.  When conducting a formal scoping or systematic review the researcher(s) determine inclusion and exclusion criteria which is determined around the research question they are trying to ask. 

 

 This criterion is applied at three times as follows:

 

·       Source Literature Database:  At this stage, the source databases are selected (i.e. which databases should be search on: PubMed, Embase, Web of Science) and the search strategy is created (note: each database will have its own unique search strategy).

·       Abstract Inclusion/Exclusion:   At this stage, the abstracts are screened based upon title and abstract based upon some criteria.  Often when a conducting a scoping review only the abstracts are used and a Full-Text review is not performed.

·       Full Text Inclusion/Exclusion:  At this stage, only the Included Abstracts are screened for their Full Text (PDF) based upon some criteria.  Only the Included Full Text articles would be used in information retrieval for a systematic review.

 

Description: Screen shot 2012-10-01 at 3.41.40 PM.png

Figure 16 - Abstract Inclusion/Exclusion

When the references are imported into Synthesis they are all marked all as “Unknown” for both Abstract and Full Text and for both the Primary and Secondary reviewer.

References can be classified as “Included”or “Excluded” (including a range of custom Excluded categories setup in Settings) in the Paper Tab under the drop down menu entitled “Include Abstract” and “Include Paper” as seen in “Figure 16 - Abstract Inclusion/Exclusion”.  This ability is applied to both the Primary and the Secondary reviewer.   

Abstracts can also be Included or Excluded by using the CRTL-I (included) or CTRL-O (out) keys for quickly classifying references.

Abstracts and the Full Text Paper can also be Included or Excluded by “first” selecting the associated drop down menu for with “Include Abstract” or “Include Paper” in the Paper Tab and using I (included) or E (excluded) keys for quickly classifying references.

Description: Screen shot 2012-10-01 at 3.41.58 PM.png

Figure 17 - Abstract Inclusion/Exclusion Filtering

Synthesis also provides the ability to see what references have been classified into four main areas: All, Unknown, Included and Excluded.  This is done by using the drop down menu found in the Reference List section of Synthesis (Figure 17 - Abstract Inclusion/Exclusion Filtering).  This will then show only the references that match one of the four areas.

 


 

Export to CSV 

Synthesis has the ability to export the information contained in the Reference List (spreadsheet like interface) to the comma separated value (CSV) format which can be imported into Microsoft Excel or other statistical software. 

 

Description: Screen shot 2012-10-01 at 3.43.49 PM.png

Figure 18 - Export to CSV

To export the contents of the Reference List, click the “Export to XLS” button in the main Synthesis window (Figure 18 - Export to Excel).

 

This will then open a new dialog window for choosing where to save the file (Figure 19 - Export to Excel File Window). 

 

The file is automatically saved with the extension “.CSV” which is associated with the Microsoft Excel 2007 program (note: other versions of Microsoft Excel can be used as well).

 

Note:  Synthesis will only export the references being shown in the reference list section.  This is affected by the search query and the primary and secondary abstract and full text inclusion/exclusion pull-down menus.  You can see the number of references being shown right under the reference list section as “displaying X references”. 

 

 

 

 

Description: Screen shot 2012-10-01 at 3.44.36 PM.png

Figure 19 - Export to CSV File Window

To import the saved file into Microsoft Excel, go to File > Open and a dialog window will open displaying all the files that are compatible with Microsoft Excel (Figure 20 - Import into Excel).  Find the file and click the “Open” button and this will open your saved Synthesis project content.

 

Description: Screen shot 2012-10-01 at 3.45.23 PM.png

Figure 20 - Import into Excel

 

The following screenshot (Figure 21 - Synthesis Data imported into Microsoft Excel) shows the Synthesis data imported into Microsoft Excel.

 

Description: Screen shot 2012-10-01 at 3.45.57 PM.png

Figure 21 - Synthesis Data imported into Microsoft Excel

 

note: When opening the file in R use the command:

synthesis <- read.csv(file="c:\\myrdata\\export1.csv",head=TRUE,sep=",")

Export to RIS (Reference Manager format)

At present Synthesis does not have the ability for cite and write that programs such as Reference Manager and EndNote.  Synthesis project databases (or subsets of the data by using the search feature and primary or secondary inclusion/exclusion filters) can be exported and imported into programs such as Reference Manager/Endnote. 

 

To accomplish this, simply click on the “Export to RIS” button located in the Reference List component (upper left section) and a dialog window will appear (Figure 22 - Export to PubMed).  Type in the name of export file and click “Save”.  This will then export the references from Synthesis in the RIS format which can be imported into EndNote, Reference Manager, RefWorks or other reference (bibliographical) software.

 

Description: Screen shot 2012-10-01 at 3.47.22 PM.png

Figure 22 - Export to RIS

For information on importing this exported file into Endnote or Reference Manager reference the section “Export to Endnote/Reference Manager”.

 

Note: Synthesis has a simple drag and drop ability to pull enabling a reference to be pulled into a word processing program such as Microsoft Word or NotePad.  The format of the drag and drop reference looks like this:  YEAR_AUTHOR_TITLE and an example of this is as follows:

Ross N. Community belonging and health. Health Rep. 2002;13(3):33-9.

Search for Missing Paper (Google Scholar Search)

Synthesis has the ability to automatically download the PDFs of the references.  However, sometimes Synthesis is unable to find those PDFs due to a variety of reason.  To help locate those missing PDFs, Synthesis has integrated a search Google Scholar feature. 

 

In the event of a missing PDF, the researcher can click the “Search Google” button in the Paper Tab section of Synthesis (Figure 23 - Google Search). 

 

Description: Screen shot 2012-10-01 at 3.47.50 PM.png

Figure 23 - Google Search

 

This button will then launch a search via Google Scholar using your default web-browser for the PDF.  Synthesis takes Google Scholar’s first “hit” and directs you to that webpage (Figure 24 - Google Scholar Journal Webpage).

 

The webpage that you are directed to may be the Journal’s webpage and not the actual PDF.  You will have to look for the PDF link and find the PDF. 

 

Description: Screen shot 2012-10-01 at 3.48.07 PM.png

Figure 24 - Google Scholar Journal Webpage

Once the PDF is located, you can save the PDF (Figure 25 - Google Scholar PDF Download) and import it into Synthesis (see section: Importing Missing PDF).

 

Description: Screen shot 2012-10-01 at 3.48.23 PM.png

Figure 25 - Google Scholar PDF Download

 


 

Import Missing Article (PDF)

In the event that Synthesis cannot automatically download the PDF for a reference, the researcher has the ability to import a missing article. 

 

This accomplished by going to the Paper tab and clicking the “Import from File” button.  This will then open a new dialog window (Figure 26 - Insert Missing PDF) where the research can select the associated PDF file and click Open to import it into the Synthesis project.

 

Description: Screen shot 2012-10-01 at 3.48.43 PM.png

Figure 26 - Insert Missing PDF

 


 

Launch External PDF Viewer

Even though Synthesis has a built in PDF Viewer, Synthesis also has the ability to launch the default PDF viewer on your computer. 

 

To launch the external default PDF viewer, simply go to the Paper tab and click the “Launch External Viewer” button.  This button will only be activated when a PDF has already been download or imported.  This will then launch the PDF viewer with the associated reference PDF.

 

Description: Screen shot 2012-10-01 at 3.47.50 PM.png

Figure 27 - Launch External PDF Viewer

Many of the PDF viewers have the ability to highlight text in the PDF and also to attach comments.  Once you have marked up the PDF you simply need to just save the PDF (not create a new PDF by using Save As) as seen in figure “Figure 28 - External PDF Viewer”.

 

The PDF viewer that we prefer for Windows as it is free and provides the functionality required is PDF XChangeViewer  (http://www.tracker-software.com/product/pdf-xchange-viewer).  For the Mac we used the default PDF viewer (Preview) as it provides annotation capabilities.

 

When you have marked up the PDF, this highlighted text and any comments made can be imported into the Synthesis project as described in the section “Import Highlighted Text and Comments from PDF”.

 

 

Description: Screen shot 2012-10-01 at 3.49.37 PM.png

Figure 28 - External PDF Viewer

 


 

Import Highlighted Text and Comments from PDF

Synthesis has the ability to imported highlighted text and comments from the PDF and import them into the comments field.  The advantage of this is that it allows the researchers to save key aspects of what they thought was import to them in the PDF.

 

To accomplish this, a researcher simply has to click the button labled “Import Highlighted Text” in the Abstract section of the Paper Tab (located in the lower left corner of Synthesis).

 

Description: Screen shot 2012-10-01 at 3.50.00 PM.png

Figure 29 - Insert Highlighted Text and Comments from PDF

The highlighted text and comments can be exported through the Bibliography tab by copying and pasting (note to Mac users: use CTRL-C and CTRL-V not Command-C and Command-V).

Secondary Reviewer

One of the major concepts with performing a Scoping or Systematic Review compared to a normal literature review is the requirement for a second reviewer.  The second reviewer is required to catch errors and oversights and reduce any bias that may occur with the project.  

 

Synthesis has fully integrated the secondary reviewer into the application.  Synthesis can be used to display the primary reviewer, the secondary reviewer or both at once.

 

To chance the reviewer, go to Project > Settings and at the top (Figure 30 - Secondary Reviewer - Settings) there are checkboxes for displaying the primary and secondary researcher’s ratings.

 

Description: Screen shot 2012-10-01 at 3.50.42 PM.png

Figure 30 - Secondary Reviewer - Settings

 

When only the “Display Secondary Researcher’s Rating” is checked and “OK” is pressed then only the Secondary reviewer information is displayed on the main Synthesis screen (Figure 31 - Secondary Reviewer - Main Screen).

 

 

Description: Screen shot 2012-10-01 at 3.50.59 PM.png

Figure 31 - Secondary Reviewer - Main Screen

 

Two components are associated with the Secondary (and Primary) reviewer. 

 

The first component is located in the Repository List (top left section of Synthesis) that allows the Abstracts and Full Text references to be displayed as either “Show All Papers”, “Show All Unknown”, “Show All Inclusions” and “Show All Exclusions”.

 

The second component is associated with the Paper Tab and used to mark the reference (Abstract or Full Text) as “Unknown”, “Included” or “Excluded”.  In addition, comments associated with the Secondary reviewer can also be added. 


 

Contrast Primary and Secondary Reviewers

Since Synthesis allows for both Primary and Secondary reviewers to classify a reference as either “Included” or “Excluded” the ability exists to quickly contrast the two reviewers. 

 

To contrast the reviewers, go to Project > Settings and click both the “Display Primary Researcher’s Ratings” and the “Display Secondary Researcher’s Ratings” checkboxes (Figure 32 - Contrast Reviewers).  This will then display both the reviewer’s information on the main Synthesis screen (Figure 33 - Contrast Reviewers - Both Included Abstracts).

 

Description: Screen shot 2012-10-01 at 3.51.28 PM.png

Figure 32 - Contrast Reviewers

 

To illustrate how the contrasting works, we would go to the Stats Tab and then toggle the reviewer’s at the top of the Synthesis main screen.  For “Figure 33 - Contrast Reviewers - Both Included Abstracts” we have set both the Primary and Secondary reviews to show the “Included” abstracts.

Description: Screen shot 2012-10-01 at 3.52.26 PM.png

Figure 33 - Contrast Reviewers - Both Included Abstracts

 

For “Figure 34 - Contrast Reviewers - Abstracts Differ” we have set the Primary reviewer to show the “Excluded” abstracts and the Secondary reviewer to show “Included” abstracts.

 

 

Description: Screen shot 2012-10-01 at 3.52.42 PM.png

Figure 34 - Contrast Reviewers - Abstracts Differ

 

Toggle On or Off the PDF Viewer

Under Settings there is now a check box called "PDF Viewer" that allows you to turn on or off the PDF Viewer located on the right side of Synthesis.

This option was added to improve the functionality when using laptops where screen real estate was limited.  A second advantage is that it improves performance (speed)when working with the spreadsheet style interface as Synthesis doesn't need to render the PDF.  

 

Description: Screen shot 2012-10-01 at 3.53.04 PM.png

Figure 35 - PDF Viewer Toggle in Settings

The following screenshot (Figure 36 - PDF viewer turned Off) shows Synthesis with the PDF Viewer turned off.

Description: Screen shot 2012-10-01 at 3.54.35 PM.png

Figure 36 - PDF viewer turned Off

 


 

Export Synthesis Project

Synthesis has the ability to export projects.  There are two main reasons to use this feature.  The first is for performing backups of your literature review.  The second reason is for working with primary and secondary reviewers using different computers to perform their literature review (see the section: Synthesis Flowchart).

 

To export a project, go to Project > Export.  This will then bring up a dialog window (Figure 37 - Export Synthesis Project) where you can enter the name of the file.  The file will be saved with the extension “.ksp” which stands for “Knowledge Share Project”.

 

For best practices, we suggest using a naming format like: PROJECT_DESCRIPTION_DATE.ksp. 

 

For example:

·       Patientflow_backup_oct2012.ksp

·       Patientflow_secondaryabstractreview_nov2012.ksp

 

Description: Screen shot 2012-10-01 at 3.56.27 PM.png

Figure 37 - Export Synthesis Project

 

 


 

Import Synthesis Project

Synthesis has the ability to export and import projects.  There are two main reasons to use the import feature.  The first reason is for recovering a backup of your literature review.  The second reason is for working with primary and secondary reviewers using different computers to perform their literature review (see the section: Synthesis Flowchart).

 

We will describe how to recover a backup Synthesis project file. The process is the same for working with primary and secondary reviewers datasets and will be noted as needed.

 

To recover a backup Synthesis project file, we first need to create a new project.  Go to Project > Change Project and then select “New Project” this will result in the following screen “Figure 38 - Create New Project”.  Type in a name for the new project.

 

 

Description: Screen shot 2012-10-01 at 3.56.50 PM.png

Figure 38 - Create New Project

 

·       To import a previous file, now go to Project > Import and select your backed up Synthesis project file.  It will have the extension “.ksp”.  You will have a screen like “Figure 39 - Import Project”.

Description: Screen shot 2012-10-01 at 3.57.07 PM.png

Figure 39 - Import Project

Once selecting your file, click on “Open” and you will get a set of options to select from when importing the file (Figure 40 - Import Project Settings).

 

 

Description: Screen shot 2012-10-01 at 3.57.27 PM.png

Figure 40 - Import Project Settings

 


 

The following options are available when Importing:

 

·       Import Project Settings.  Existing Project Settings will be erased.

·       Import Primary Researcher’s Ratings.  Existing Primary Researcher’s Rating will be erased.

·       Import Secondary Researcher’s Ratings.  Existing Secondary Researcher’s Rating will be erased.

·       Import Custom Columns Data. Most recent data will overwrite older data.

 

For recovering a backup, select all the checkboxes. 

 

When working with the primary and secondary reviewers, select the required checkboxes (be careful you are not overwriting your primary data).

 

Important: If you are working with Primary or Secondary reviewer import – make sure you have backed up the Synthesis project that you are importing into.

 

Once the Import has occurred you will get a status of the references being imported (Figure 41 - Import Project Settings Status).

 

Description: Screen shot 2012-10-01 at 3.57.44 PM.png

Figure 41 - Import Project Settings Status

You can select Project > Settings to ensure that all the settings were imported correctly (Figure 42 - Synthesis Project Settings).

 

Description: Screen shot 2012-10-01 at 3.58.11 PM.png

Figure 42 - Synthesis Project Settings

After importing, Synthesis will have all of the data as the original backup (Figure 43 - Synthesis Main Screen).

 

Description: Screen shot 2012-10-01 at 3.58.31 PM.png

Figure 43 - Synthesis Main Screen


 

Synthesis Tools - Kappa Calculator

Synthesis has the ability to perform a Kappa Calculation on two columns.  This allows a user to quickly see what level of agreement (inter-rater agreement) there is between two columns. 

To access this feature, go to Tools > Kappa Calculator.

Select the two columns to be compared and click the Calculate button.

Figure 44 - Kappa Calculator

note: Columns need to be filled in using a binary classification such as: Yes/No(Blank); True/False(Blank); T/F(Blank).  The negative classification (i.e. No/False) can be left blank.

note: The algorithm that Synthesis applies is Cohen's Kappa[1].

note: When the level of agreement between the two columns are very close, the Kappa Score can have a negative value.


 

 

Kappa

Agreement

<0

Less than chance Agreement

0.01-0.20

Slight agreement

0.21-0.40

Fair Agreement

0.41-0.60

Moderate Agreement

0.61-0.80

Substantial Agreement

0.81-0.99

Almost Perfect Agreement

Table 1 - Kappa Agreement[2]

 

 

 


 

Synthesis Tools - Sample Column

Synthesis has the ability to create a random sample for a column.  This allows the user to create a column with a specific percentage, radmonized, from the rows being displayed in Synthesis.  The reason for this feature, is sometimes a user may want to further investigate a 10% random sample for quality insurance or it may be determined that a secondary reviewer only review 20% of the articles. 

To access this feature, go to Tools > Sample Column

Use the slider to select the X percentage to be randomly selected.  Once pressing OK, Synthesis will create the column and insert "Yes" in the X% of the rows.  

Figure 45 - Sample Column

To use the random sample in the search filter use:

custom_Sample2:Yes

 


 

Synthesis Tools - Grey Literature

Synthesis has the ability to add other documents to a project.  This is known as Grey Literature within Synthesis as the references come from non-indexed bibliographical sources. 

Grey Literature is where the user could add reports, internal documents or other sources of information.  Note:  Grey Literature is only able to attach PDF documents. 

Additionally, the Grey Literature component in Synthesis allows a user to copy and paste BibTex information as well.  This is extremely useful for adding references from ACM (http://librarians.acm.org/digital-library) or book references.

To access this feature, go to Tools > Grey Literature

Figure 46 - Grey Literature

To Add a new grey literature reference simply click on New and enter in the information.  Other buttons include Edit and Delete.

Any Grey Literature references will have the Source Database (displayed under the Stats Tab) as "Synthesis Grey Literature".

Synthesis also allows BibTex references to be added via the Paste Button.  This will open a new Window and paste anything in the clipboard into the text window.  Click Import to add.

Figure 47 - Grey Literature BibTex import

 

Note: OttoBib.com allows you to enter a ISBN (identifier of books) and creates a BibTex entry of the reference that can be imported into Synthesis using the Grey Literature Paste button.  OttoBib also has a Google Chrome plug-in that can be added to the Google Chrome web-browser to make this task easier.  To find the ISBN of a book you can search for the book in Amazon.com which will show the ISBN under the Book Description.

 


 

Synthesis Tools - Scan for PDFs

Synthesis has the ability to search for missing PDFs from a project in all other projects contained in the /LiteratureReview folder.  Since many projects may overlap for a researcher, the PDF may have already been downloaded. 

To access this feature, go to Tools > Scan for PDFs.

Figure 48 - Scan for PDFs

 

 

 

 


 

Synthesis Tools - Extract PDF Images

Synthesis has the ability to extract the PDF images contained in a Project's PDFs.  This allows a user to quickly see how information is visualized in the project.

To access this feature, go to Tools > Extract PDF Images.

Synthesis will then scan the project directory PDFs and place all the images as jpeg files in the \LiteratureReview\<project>\images folder.

Figure 49 - Extract PDF images

Figure 50 - Figures from PDFs

Note: When working with the images, it may be best to delete any images less than 15kb as these images are most likely logos, copyright notices, etc. 


 

Preferences - ezProxy

ezProzy is a protocol for accessing your institutions library journal subscriptions from an outside location such as your home. 

 

To access this feature, go to Project > Preferences.

Figure 51 - ezProxy

 

The following information will need to be entered:

 

Proxy URL (in the following format):

http://ezproxy.lib.ucalgary.ca:2048/login?user=$(user)&pwd=$(pwd)&url=$(url)

 

note: in many cases the institutions ezProxy will be simply in the following format: " http://ezproxy.lib.ucalgary.ca".  If this is the case appended the following information: "/login?user=$(user)&pwd=$(pwd)&url=$(url)",  so the Proxy URL looks like the following: 

http://ezproxy.lib.ucalgary.ca:2048/login?user=$(user)&pwd=$(pwd)&url=$(url)

 

User$(user):

Your institution username

 

Password$(pwd):

Your institution password

 

 


 

Register Synthesis Software (User License)

Synthesis is licensed on a per computer and per duration basis. 

 

Synthesis is usually licensed on a yearly basis.  Synthesis is also tried to the computer hardware, so this means that a license for Synthesis will only be valid for that computer and cannot be used on another computer.  

 

The default demonstration (non-licensed) version of Synthesis only has the ability to import 200 references into a Synthesis Project.  This allows the future-user of Synthesis to try the various features of Synthesis before having to purchase a license. 

 

When Synthesis is licensed the restriction of only being able to import 200 references into a project is removed.  Once a license has expired, the user is still able to use Synthesis with their project files – but is not able to import more than 200 references into a project.

 

To register for a license you need to first install Synthesis on your computer and (Figure 44 - Register Synthesis Software):

·       Go to Help > Register

·       Email the Installation ID to sales@synthesis.info

 

Description: Screen shot 2012-10-01 at 3.58.58 PM.png

Figure 52 - Register Synthesis Software

 

When sales@synthesis.info has received your Installation ID they will email you back a Registration Key based upon your license.  Enter this Registration Key into the Registration Key field and click okay.  This will also tell you for how long your license is valid.

Terms of the Synthesis license agreement can be found in the section “Synthesis User License Agreement”.

Note Mac Users: copying and pasting using the Command-C and Command-V keys will not work.  Use the Ctrl-C and Ctrl-V keys for copying and pasting. 

 

 


 

importing and de-duplicating references

De-duplicating references is a tricky problem.  It seems that each bibliographical database such as Embase, Web of Science, PubMed, etchas a different way of entering the journal articles into their databases.  Some of the issues may be slight spelling mistakes, entering a symbol such as "<=" in one database and writing "less than or equals to" in another database, or very small variations between references such as using the same abstract, but changing one thing such as the year "we report on data from 1989".

Synthesis also wanted to address the updating of literature reviews.  According to the Cochrane Collaboration, Cochrane Intervention reviews should be updated within two years as "Systematic reviews that are not maintained may become out of date or misleading"   (source: The Cochrane Collaboration. “The Cochrane Book Series” Imprint.Published by John Wiley & Sons, Ltd. 2008).

How Synthesis handles the importing of references is that it identifies references three ways. 

·       Abstracts Imported - these are new abstracts that are imported into your Synthesis project file.

·       Precise matches (skipped) - these are abstracts that are 100% similar to an abstract already in your Synthesis project file and that come from the same source database (i.e. 1st abstract came from PubMed and 2nd Abstract came from PubMed).  This allows you to update the Synthesis project file and exclude all the previous imports from the same library database (i.e. PubMed).

·       Duplicate Abstracts Imported - these are all the duplicates that are no precise matches (not from the same source library database or that are 99% similar to a previously imported reference) 

For example, if we were to import a reference file from PubMed using the query "National Health Interview Survey" (extracted September 12, 2012) we would get the following (the right column is showing information contained in the Stats tab after importing):

 

If you were to re-import the exact same reference file you would get the following (the right column is showing information contained in the Stats tab after importing):

Note: the import takes longer to import the second time as Synthesis needs to compare the reference being imported with all the other abstracts already imported into the Synthesis project file.

Notice, that the Unique Papers are the same, however, Excluded Duplicates have increased (note: to be corrected in a future release of the software).

 

 

Verification of Duplicates

Once the bibliographic references have been imported into Synthesis it is recommended to verify the automated duplicates identification.  Because there is so much varition in how the abstracts are entered, the automated algorithm may identify references as duplicates when they are not.  An example of this may be a “Part I” “Part II” reference, where the authors, title and abstract are exactly the same, except for one additional character.  In this case, the “Part II” refrence would be identified as a duplicate.  The reason why the algorithm works this way is that for some bibliograpical databases they often add a copyright statement at the end of the abstract or may use words in stead of charcters such as “less than” instead of “<”.

Synthesis addresses this by allowing the user to verify the duplicates and toggle “on” or “off”.

The process for doing this is:

·       Go into settings

o   Turn on “Duplicate Mode”

o   Exit Settings

·       Main Window

o   The Duplicate column now appears

o   Sort the references by “Title”

o   Scroll down the references and verify that Duplicates are correctly identified.  Also, check if any papers may have been missed

§  Use Crtl-I and Ctrl-O to mark as Duplicate or Not.

 

Figure 53 - Settings Window - Duplicates

 

Figure 54 - Synthesis Main Screen - Duplicates

importing and de-duplicating references - Case Study

To illustrate how Synthesis imports references from multiple library databases, we will conduct the following example.  We searched for the term "patient flow" on the following library databases and imported them into Synthesis:

·       PubMed

·       Ovid Medline

·       Ovid Embase

·       Web of Science (note: can only export 500references at a time)

·       IEEE

·       Scopus

·       ProQuest (note: you can only export 2,000 references at a time)

·       EndNote

·       Grey Literature (BibTex)

 

At each import, Synthesis shows how many references were imported and how many were duplicates.  Note that none of the imports had any references that fell under "Precise matches (skipped)" as they all came from different library databases.

PubMed

 


 

MedLine

 

Embase

 

IEEE

 

Scopus

ProQuest

 

Web of Knowledge

 


 

Supported Source Databases

 

The following are the source databases that Synthesis has been tested with.  As there are many databases support is not guaranteed for all. 

In the case that a source database that you want to use is not referenced in the list (Synthesis Version 2.3 Import Validation Check) we recommend that you try to export the references in the “pubmed” or “medline” format.

 

Synthesis Version 2.3 Import Validation Check

 

Database

 

Import into Synthesis

Fields

 

Export to EndNote

Source

Issue

PubMed

Ovid Medline

 

 

Ovid Embase

 

 

Web of Science

 

 

Scopus

 

 

IEEE

 

 

ProQuest

 

 

 

ACM

 

 

 

EndNote

 

 

 

BibTex

 

 

 

Grey Literature

 

 

 

 

 

 

Importing from PubMed

Synthesis can import references from http://pubmed.org. 

In the following example we used the search term:

"national health interview survey" or "NHIS"

This search returned 2229 results (executed July 17, 2012).  To import these references into Synthesis you need to:

·       Select "Send To"

·       Select File

·       Choose the format to be Medline

·       Click Create File

 

Figure 55 - PubMed

This will then save a file called “pubmed_results.txt”.  Import this file in Synthesis the same way you would any other references. 

Importing from Ovid Medline/Embase/Etc

Synthesis can import references from Ovid.  Ovid contains many different databases such as Medline, Embase, Global Health and several others. 

In the following example we used the search term:  "patient flow"

This returned 1,099 results.  To export these references we would select the references, in this case by checking the “All” checkbox at the bottom of “Figure 46 - Ovid”.

Figure 56 - Ovid

This will then open a new window on the webpage as seen in “Figure 47 - Ovid Export”.

 

Figure 57 - Ovid Export

To export to Synthesis use the following settings:

·       Export To:  select “Reference Manager”

·       Select Fields to Display: select “Complete Reference”

·       Include: turn OFF all check boxes

·       Click Export Citations

This will then save a file called UNKNOWN.  Import this file in Synthesis the same way you would any other references.

Importing from Web of Knowledge

Synthesis can import references from Web of Knowledge (previously Web of Science).

After conducting your search -- you will get a screen similar to this:

Figure 58 - Web of Knowledge top of webpage

Go to the bottom of the page and enter in the number of records to download (note: you can only download 500 at  a time).  Then select Full Record.  You will then press the EndNote botton (EndNote is owned by the same company as Reference Manager and Synthesis can import both of their formats). 

This then saves a file called savedrecs.ciw.  Import this file in Synthesis the same way you would any other references. 


 

Figure 59 - Web of Knowledge bottom of webpage

Note: Web of Knowledge (i.e. Web of Science) will not have any Mesh headings.  Although they have good DOI information which helps in the automated downloading of PDFs.

 


 

Importing from IEEE

Synthesis can import references from the IEEE Xplore Digital Library.

After conducting your search -- you will get a screen similar to this:

Figure 60 - IEEE Xplore Digital Library

To export the references:

·       Click on the webpage button “Select All on Page” to choose your references

·       Click on the webpage button “Export Results”.  This will then launch a dialog window. 

·       Under Include: select the “citation & Abstract” option

·       Under Format: select the “EndNote, ProCite, RefMan” option

·       Click the “Download Citation” button.

This then saves a file called UNKNOWN.  Import this file in Synthesis the same way you would any other references. 

Importing Scopus (Elsevier)

Synthesis can import references from the Scopus (Elsevier) database.

After conducting your search -- you will get a screen similar to this:

Figure 62 - Scopus (Elsevier)

To export the references:

·       Click the “All” checkbox – to select all references

·       Click the “Export” link.  This will then go to a new webpage as seen with “Figure 52 - Scopus (Elsevier) Export”. 

·       In new WebPage entitled “Output: Export, Print, Email or Create a Bibliography” coose the following options

·       Export Format: Select the “RIS Format (Reference Manager, ProCite, EndNote)

·       Output: Select the “Abstract format”

·       Click the “Export” button.

 

Figure 63 - Scopus (Elsevier) Export

 

This then saves a file called UNKNOWN.  Import this file in Synthesis the same way you would any other references. 

Importing ProQuest

Synthesis can import references from ProQuest which provides a number of different source databases such as ones related to various newspapers or dissertations.

After conducting your search -- you will get a screen similar to this:

Figure 64–ProQuest

To export the references, click on the “Export/Save” link as seen in “Figure 53 – ProQuest” and then select the “EndNote, Reference Manager or ProCite” option.

This will then launch a new dialog window as seen in “Figure 54 - ProQuest Export”.

 

Figure 65 - ProQuest Export

Under Content select the “Citation, abstract, indexing” option and click the “Continue” button.

This then saves a file called UNKNOWN.  Import this file in Synthesis the same way you would any other references. 

Note: You can change ProQuest to display 100 results per page.  This allows you to quickly select 100 references at a time to add to your collection.  Also, Proquest will only export 2,000 references at one time.

Note: Required running on Firefox compared to Google Chrome -- as it needs to open a pop-up window in order to download the file.

 


 

Importing EndNote

 

[To Do]

 

 

 


 

Importing Grey Literature (BibTex)

Synthesis also allows BibTex references to be added via the Paste Button in the Grey Literature component. 

To access this feature, go to Tools > Grey Literature.

In the Grey Literature window, click on the Paste button.  This will open a new Window and paste anything in the clipboard (i.e. your BibTex refrence) into the text window.  Click Import to add.

Figure 66 - BibTex Import

 

Note: OttoBib.com allows you to enter a ISBN (identifier of books) and creates a BibTex entry of the reference that can be imported into Synthesis using the Grey Literature Paste button.  OttoBib also has a Google Chrome plug-in that can be added to the Google Chrome web-browser to make this task easier.  To find the ISBN of a book you can search for the book in Amazon.com which will show the ISBN under the Book Description.

 

 

searching in Synthesis

Synthesis provides a very powerful query language for searching the Synthesis references.  Not only can Synthesis search the titles and abstracts, nut it can also search the full text PDF document as well.  When Synthesis does do PDF search, the words are also highlighted in the PDF document as well.

A breakdown of the Query Search syntax is as follows:

·       To query multiple words use quotations

o   "Behavior Risk Factor Surveillance System"

·       Boolean operators (must be capitalized)

o   AND

§  "diabetes" AND "hypertension"

o   OR

§  "diabetes" OR "hypertension"

o   NOT

§  "diabetes" NOT "hypertension"

·       + (includes documents with the term)

o   +"goodness of fit"

·       - (excludes documents with the term)

o   -"goodness of fit"

·       Wildcard Searches

o   diabet* (returns diabetes, diabetic, etc)

·       ? can be used for a single character such as "Pub?ed"

·       Proximity Search - returns results where health and survey are within 10 words

o   "health survey"~10

·       You can also focus your search to specific fields in the database.  The following fields are supported (field names must be in lower case):

o   title:"health professional"

o   abstract:"National Health Interview Survey"

o   year:2011

o   author:"Yoo"

o   journal:"MMWR"

o   source:"pubmed"

o   mesh:"HIV"

o   fulltext:"simulation"

o   havepdf:Yes (references where there is a PDF)

o   primary_include_abstract:Included or Excluded

o   secondary_include_abstract:Included or Excluded

o   primary_include_fulltext:Included or Excluded

o   secondary_include_fulltext:Included or Excluded

·       Range

o   year:[2010 TO 2012]

·       To query when the abstract is empty:    

o   title:[a* TO z*] -abstract:[a* TO z*] 

o   title:[a* TO z*] AND NOT abstract:[a* TO z*] 

·       To query when there is a PDF:

o   fulltext:[a* TO z*] 

·       Escape characters () must be preceded with \ if the characters are used in a query:

·       Lucene uses these characters: + - && || ! ( ) { } [ ] ^ " ~ * ? : \

o   "how many\?"

·       Grouping

o   ("Canadian" or "Canada") AND "Hypertension"

o   title(+"Survey" +"Hypertension")

o   +"BRFSS" -"California" "Ohio"

 

 

 


 

external applications for Synthesis

 

Synthesis can be integrated with the following external applications:

 

·       RefWorks ( bibliography software)

·       EndNote ( bibliography software)

·       Reference Manager ( bibliography software)

·       Latex/Sweave via BibTex

·       Carrot2 (Text Clustering Software)

·       WebExcel

·       SmartColumns

 

Primary external applications (covered in the Tutorial section):

·       PDF Viewer

·       Microsoft Excel

·       Google Scholar

 

 


 

Export to RefWorks

RefWorks is a cite and write web-based application that many institutional libraries subscribe to.  RefWorks functions by storing all references on a web-based server, where you can upload your references, and then by syncing with a Microsoft Word plug-in that supports both Windows and the Mac.

 

To export to RefWorks simply click the "Export to RIS" button on the Main Screen.  

 

This will then open a new window for you to save the filename (i.e. Synthesistorefworks). 

The file created will have the extension ".ris" (i.e. Synthesistorefworks.ris).  This file will be saved using the NLM PubMed format.

To import into RefWorks, log in to the RefWorks web-site.

 

Click on Import (right hand side of the RefWorks webpage) and select the following configuration:

 

Figure 67 - Import into RefWorks

 

Import Filter/Data Source:          NLM PubMed

Database:                                     PubMed

Select Text File:                           the RIS file that you created

 

Import Into Folder:

Note: It is worthwhile to create a folder for each project to keep your references organized.

 

Once the references have been imported in the RefWorks website, you can Sync the RefWorks plug-in in Microsoft Word to update the references.

Export to EndNote/Reference Manager

To export to EndNote (or Reference Manager) you simply click the "Export to RIS" button on the Main Screen.

Note: Synthesis will only export the references that are currently displaying (dependent on if you are viewing "Show All Papers", "Only Unknown", etc and also what Search Filters you currently have executed).

This will then open a new window for you to save the filename (i.e. Synthesistoendnote). 

The file created will have the extension ".ris" (i.e. Synthesistoendnote.ris).  This file will be saved using the NLM PubMed format.

Figure 68 - Export to EndNote

 

Figure 69 - Export to EndNote - Save Window

 

ENDNOTE

Open EndNote into your project file.  Select File > Import which will open a new window.  Select your import file (i.e. Synthesistoendnote.ris) and under "Import Option" select "PubMed (NLM)" and click Import.  This will then import your Synthesis file.

Figure 70 - Importing into EndNote

 


 

REFERNCE MANAGER

 

Open Reference Manager into your project file.  Select File > Import Text File which will open a new window.  Select your Text File (i.e. Synthesistoendnote.ris) and under "Filter" select "PubMed (NLM)" and click Import.  This will then import your Synthesis file.

 

Description: http://www.refman.com/images/imgPM7.gif

Figure 71 - Importing into Reference Manager

 

Source: http://www.refman.com/support/rmpmigm.asp

Using BibTeX in Latex/Sweave

Once you have exported the BibTeX references from the SynthesisBibTeX Tab you can include your bibliology in Sweave (Latex).  In this example we are working with Sweave within RStudio.

Create a file in your R working directory called testbib.bib.  In this file you can save BibTeX citations.  An example of the some content is below:

BibTeX CITATIONS

@article{wun2007,

author = {WunLM,Ezzati-Rice TM,Diaz-TenaN,Greenblatt J},

title = {On modelling response propensity for dwelling unit (DU) level non-response adjustment in the Medical Expenditure Panel Survey (MEPS).},

journal = {Stat Med. 2007 Apr 15;26(8):1875-84.},

year = {2007},

}

 

@article{theis2007,

   author = {TheisKA,MurphyL,HootmanJM,HelmickCG,Yelin E},

title = {Prevalence and correlates of arthritis-attributable work limitation in the US population among persons ages 18-64: 2002 National Health Interview Survey Data.},

journal = {Arthritis Rheum. 2007 Apr 15;57(3):355-63.},

year = {2007},

}

 

@article{2008:2,

title = {Persons tested for HIV--United States, 2006.},

journal = {MMWR Morb Mortal Wkly Rep. 2008 Aug 8;57(31):845-9.},

year = {2008},

}

 

@article{kennedy2008,

author = {Kennedy J,WangCC,Wu CH},

title = {Patient Disclosure about Herb and Supplement Use among Adults in the US.},

journal = {Evid Based Complement Alternat Med. 2008 Dec;5(4):451-6. Epub 2007 May 17.},

year = {2008},

}

 

@article{jones2008,

author = {Jones GC,Sinclair LB},

title = {Multiple health disparities among minority adults with mobility limitations: san application of the ICF framework and codes.},

journal = {DisabilRehabil. 2008;30(12-13):901-15.},

year = {2008},

}

 

 

SWEAVE

 

In Sweave the basic outline is as follows:

 

 

\documentclass{article}

\bibliographystyle{plain}

 

\usepackage{amsmath}

\usepackage{amscd}

\usepackage[tableposition=top]{caption}

\usepackage{ifthen}

\usepackage[utf8]{inputenc}

 

\begin{document}

 

\title{BibTex Demo}

\author{Healthcare Simulations Inc}

\maketitle

 

The following references are included \cite{2008:2} and \cite{kennedy2008}.

 

See \cite{theis2007} for details.

 

\bibliography{testbib}

 

\end{document}

 

The following PDF is created:

 

Note: To apply superscript for reference use the following commands:

\usepackage{overcite}

\bibliographystyle{plain}

 


 

Using Synthesis with Carrot2 (Clustering Engine)

Carrot2 is a text classification (clustering engine) that classifies documents into clusters.  This is mainly an unexplored method in the area of medical scoping reviews. 

Synthesis project files can be accessed through the Carrot2 application (note: there is no reason to export a Synthesis project).

The Carrot2 application has essentially 4 components.  The first component is accessing the Synthesis project database – this is located in the upper left hand corner of Carrot2.  The second component is the Text Clustering categories which are displayed in the middle of the Carrot2 application.  The third is a visualization component located in the lower left hand corner of the application.  Finally, located on the right side of Carrot2 are parameters that can be adjusted around the clustering algorithm(s).

Carrot2 can be downloaded at: http://project.carrot2.org/download.html

We have also modified Carrot2 to allow the researcher to remove words/concepts from the clustering algorithm (in case they don’t make sense or are duplicated with other words/concepts).

Figure 72 - Carrot2 Application

The following is the configuration parameters for accessing the Synthesis project databases.


Note: Once change are made  click the "Process" button.

Source: You will need to set this to Lucene, as Synthesis uses the Lucene database engine

Algorithm: There are several clustering algorithms available in Carrot2.  The most reliable from our experience is Lingo.

Document content field: This is the field that Carrot2 will actually cluster on.  The most common ones to use with Synthesis databases are: abstract,

Document title field: This is the field that Carrot2 will display in the Document results.

Index Directory: This is the location of the Synthesis project lucene database.  It should be in the format: c:\users\YOURNAME\literaturereview\YOURPROJECT\index

Query: A Lucene query is required.  Good default ones:

·       mesh:"humans"   (to cluster all MESH headings)

·       abstract[A* TO Z*] (to cluster all abstracts)

·       mesh:"humans" +fulltext:[a* TO z*]   (to cluster PDF)

Results: This field indicates how many results to return.  this should be larger than the references in your Synthesis project.

Note: There is currently no way to only cluster on "Included" references. 

The following is one of the visualizations that Carrot2 offers.

Figure 73 - Carrot2 Application Cluster Map

Note: There is currently no way to only cluster on "Included" references. 

Using Synthesis with HSI WebExcel

An external application developed by Healthcare Simulations Inc (HSI)  isable to import the Excel spreadsheet created by Synthesis and automatically create charts on every column of data.  This provides the researcher with a quick understanding of the data.

The charts can be altered such as changing the chart type (Pie, Line, Bar, etc) and sub-grouping the information.

WebExcel can also perform filtering on the data through the use of data driven drop down menus. 

Figure 74 - WebExcel

WebExcel also has an R component and the possibly exists to have WebExcel perform the meta analysis and produce all of the statistics and charts (i.e. ForestPlot).

 


 

Annotate Visible Columns

Synthesis has the ability to create custom columns based upon the references being displayed.

Once a search has been performed (in this example search for the term "confidence interval") the user can click the "Annotate Visible Rows" button.

Figure 75 - Annotate Visible Columns

This will then launch a window where the custom column parameters can be entered.

Figure 76 - Annotation Column Definition

There are two ways to create a custom column:

·       Create a new column

o   This option is good for creating columns with Yes/No values such as a specific country (i.e. Canada) or specific type of regression (i.e. Logistic).

·       Use an Existing Column where values can

o   This option is good when you have multiple possible values such as "Countries" or "Types of Regression".

 

The "Value on match" attribute is what value you want assigned to the column.

Once the parameters have been selected and the user presses OK, a column is either created or exisiting column is used and filled in with the corresponding values.

Figure 77 - Annotated Columns

Note: to search using the custom column, use the format custom_<ColumnName>:<value>  (i.e. custom_CI:Yes)

Synthesis Project File Folder

 

All the Synthesis project files are stored in the folder:

 

Windows:           C:\Users\YOURNAME\LiteratureReview

Mac:                   \Users\YOURNAME\LiteratureReview

 

In the LiteratureReview directory you will find a list of project directories.  A directory is created for each project created in Synthesis.  For example: 

\Users\YOURNAME\LiteratureReview\ProjectA

\Users\YOURNAME\LiteratureReview\ProjectB

 

Inside each Project directory will be the associated PDFs that are attached to the folder.  There are in the format: YEAR_AUTHOR_TITLE.pdf.

 

You can copy these PDF files – but do not remove or delete them.

 

Note: You cannot add PDFs to the Synthesis by simply placing them in the directory – they must be attached to the reference through the Synthesis application.

 

Inside each Project directory there are the following directories:

\index

“Index”.  This is the Synthesis project database – do not change/modify/delete this directory as you will lose all your information.

\import

"Import".  Synthesis keeps the original reference files imported with a date/time stamp of when the reference file was imported. 

\images

"Images". If the Tools > Extract PDF Images has been run, then all images extracted are in this directory.

 

 

 

 

Working with Synthesis Flowchart

 

Backing up Synthesis Projects

 

 There are two ways to backup Synthesis files.  The first method is to perform the backup within the Synthesis application.  The second method is to make a backup of the file folder (directory) where Synthesis stores the project data.

 

We suggest that backups are used/preformed for the following situations:

 

·       For Standard Backups purposes.

·       Before Importing Projects such as when merging the primary and secondary reviewer databases.

 

Backing up Synthesis Files within the Synthesis Program

 

To make a backup, use the Project > Export function in Synthesis.This will create a “.KSP” (Knowledge Share Project) file. 

 

For best practices, we suggest you name the backup file with a meaningful name.  We personally use the format:

PROJECT_DESCRIPTION_DATE.ksp

 

Some examples are:

             Machinelearning_fullback_nov_2012.ksp

             Machinelearning_primaryreviewer_nov_2012.ksp

 

 

To retrieve your backup, you will need to create a new project in Synthesis and use the Project > Import function to import the file.  Since, you want to retrieve everything from the backup ensure that all checkboxes are checked as seen in “Figure 62 - Importing Synthesis Project”.

 

Description: Screen shot 2012-10-01 at 3.57.27 PM.png

Figure 78 - Importing Synthesis Project

 

Backing up Synthesis Files through the File Manager

 

All the Synthesis project files are stored in the folder:

 

Windows:           C:\Users\YOURNAME\LiteratureReview

Mac:                   \Users\YOURNAME\LiteratureReview

 

To backup ALL of the Synthesis project files, simply backup the entire LiteratureReview file.

 

If you only want to backup a specific file, go into the LiteratureReview directory and you will find a list of project directories.  Simply make a backup of the specific directory you require.

 

To retrieve your backup, you simply need to copy the directory back to its location. 

 

In the case of having backed up your entire LiteratureReview directory, place this in your \Users\YOURNAME\ directory. 

 

In the case of having backed up a specific project, place this in your \Users\YOURNAME\LiteratureReview directory. 


 

Synthesis User License Agreement

 

 This Software License Agreement ("Agreement") is made and effective upon download of the software or registration of the license key between Healthcare Simulations Inc("Developer") and the Licensee ("Licensee").  Developer has developed and licenses to users its software program marketed under the name KnowledgeShare Version 2.X (the "Software").  Licensee desires to utilize a copy of the Software.

 NOW, THEREFORE, in consideration of the mutual promises set forth herein, Developer and Licensee agree as follows:

1. License. Developer hereby grants to Licensee a limited license to use the Software as set forth in this Agreement and through the license the Licensee has acquired through thewww.healthcaresimulations.com website or as mutually agreed to via email with the Developer.

2. Restrictions. Licensee shall not modify, copy, duplicate, reproduce, license or sublicense the Software, or transfer or convey the Software or any right in the Software to anyone else without the prior written consent of Developer; with the exception (a) one copy for backup, and (b) one copy for testing.

3. Fee. In consideration for the grant of the license and the use of the Software, Licensee agrees to pay Developer the cost of the license specified on the Synthesis webpage on the www.healthcaresimulations.com website or as mutually agreed to via email with the Developer.

4. Warranty of Title. Developer hereby represents and warrants to Licensee that Developer is the owner of the Software or otherwise has the right to grant to Licensee the rights set forth in this Agreement. In the event of any breach or threatened breach of the foregoing representation and warranty, Licensee's sole remedy shall be to require Developer refund to Licensee the full amount of the license fee upon the return of the Software and all copies thereof to Developer.

5. Warranty of Functionality. A. The Software is provided as part of a research project.  No warranty is provided and the Developer will not be liable for any damages.

6. Software Maintenance. A. Developer shall provide to Licensee any new, corrected or enhanced version of the Software as created by Developer for 1 year from the date of the license being issued. Such enhancement shall include all modifications to the Software which increase the speed, efficiency or ease of use of the Software, or add additional capabilities or functionality to the Software, but shall not include any substantially new or rewritten version of the Software.

7. Payment. Payment of the license fee shall be made upon delivery of the Software.

8. Warranty Disclaimer. DEVELOPER'S WARRANTIES SET FORTH IN THIS AGREEMENT ARE EXCLUSIVE AND ARE IN LIEU OF ALL OTHER WARRANTIES, EXPRESSED OR IMPLIED, INCLUDING BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE.

9. Limitation of Liability. Developer shall not be responsible for, and shall not pay, any amount of incidental, consequential or other indirect damages, whether based on lost revenue or otherwise, regardless of whether Developer was advised of the possibility of such losses in advance. In no event shall Developer's liability hereunder exceed the amount of license fees paid by Licensee, regardless of whether Licensee's claim is based on contract, tort, strict liability, product liability or otherwise.

10. Notice. Any notice required by this Agreement or given in connection with it, shall be in writing and shall be given to the appropriate party by personal delivery or by certified mail, postage prepaid, recognized overnight delivery services or email (with receipt from the sender).

If to Developer:

Healthcare Simulations Inc Calgary, Alberta, Canada

11. Governing Law. This Agreement shall be construed and enforced in accordance with the laws of the province of Alberta.

12. No Assignment. Neither this Agreement nor any interest in this Agreement may be assigned by Licensee without the prior express written approval of Developer.

13. Final Agreement. This Agreement terminates and supersedes all prior understandings or agreements on the subject matter hereof. This Agreement may be modified only by a further writing that is duly executed by both parties.

14. Severability. If any term of this Agreement is held by a court of competent jurisdiction to be invalid or unenforceable, then this Agreement, including all of the remaining terms, will remain in full force and effect as if such invalid or unenforceable term had never been included.

15. Headings. Headings used in this Agreement are provided for convenience only and shall not be used to construe meaning or intent.


 

APPENDIX a - KEYWORDS

 

The following are some generic keywords that can be used in the Abstract and Full Text keyword section in Settings.

Introduction

Background

Methods

Results

Conclusion

Discussion

Summary

 

p-value

pvalue

“p<0.05”

 

 

 

 

 

 

 


 

INDEX

 


Annotate, 9, 102

backup, 51, 52, 54, 55, 106, 107, 108

Bibliography, 18, 29, 44, 81

BibTeX, 30, 95

Boolean operators, 87

Carrot2, 89, 98, 99, 100

CSV, 9, 35, 36

drag and drop, 26

Duplicates, 9, 18, 27, 31, 67, 68, 69, 70

Embase, 16, 28, 33, 66, 71, 72, 74, 76

EndNote, 9, 10, 38, 71, 74, 78, 80, 81, 83, 85, 89, 92, 93

Exclusion, 20, 21, 33, 34

Export Synthesis Project, 51

Export to RIS, 9, 38, 90, 92

Extract PDF Images, 62

ezproxy, 23, 63

ezProxy, 23, 63

Google Scholar, 23, 39, 40, 89

Grey Literature, 10, 59, 60, 71, 74, 86

IEEE, 71, 72, 74, 80

Import from File, 23, 41

Import Highlighted Text, 42, 44

Import Synthesis Project, 52

Inclusion, 33, 34

Kappa, 27, 56, 57

Latex, 30, 89, 95

Medline, 14, 71, 74, 75, 76

Microsoft Excel, 35, 36, 37, 89

Mircosoft Excel, 101

Ovid, 71, 74, 76, 77

PDF Viewer, 10, 18, 22, 42, 43, 49, 89

ProQuest, 71, 73, 74, 83, 84

Proximity Search, 87

PubMed, 10, 14, 15, 16, 28, 33, 38, 66, 71, 74, 75, 90, 91, 92, 93, 94

Query, 87, 99

Reference Manager, 38, 77, 78, 81, 83, 89, 92, 94

RefWorks, 9, 38, 89, 90, 91

RIS, 38, 81, 91

Sample Column, 58

Scan for PDFs, 61

Scopus, 71, 72, 74, 81, 82

Secondary Reviewer, 17, 45, 46

Settings, 8, 9, 18, 20, 34, 45, 47, 49, 53, 54, 55, 69, 111

Source Database, 10, 28, 60

Web of Knowledge, 73, 78, 79

Web of Science, 33, 66, 71, 74, 78, 79

WebExcel, 89, 101

Wildcard Searches, 87

Word Cloud, 10, 32


 



[1] http://en.wikipedia.org/wiki/Cohen's_kappa

[2] Viera A, Garrett JM; Understanding Interobserver Agreement: The Kappa Statistic. Fam Med 2005; 37(5):360-3.