Main Page
From VxOware
Documentation for VxOware
Contents |
1. News
- The source code is available at: https://sourceforge.net/projects/vxoware/.
- A continuous build is located at http://virbo.org/meta/.
2. Abstract
The recent Heliophysics Virtual Observatory (VxO) effort involves the development of separate observatories with a low overlap in physical domain or area of scientific specialization and a high degree of overlap in metadata management needs. VxOware is a content and metadata management system. While it is intended for use by a VxO specifically, it can also be used by any entity that manages structured metadata. VxOware has many features of a content management system and extensively uses the W3C recommendations for XML (Extensible Markup Language), XQuery (XML Query), and XSLT (Extensible Style Sheet Language Transformations). VxOware has features such as system and user administration, search, user-editable content, version tracking, and a wiki. Besides virtual observatories, the intended user-base of VxOware includes a group or an instrument team that has developed a directory structure of data files and would like to make this data, and its associated metadata, available in the virtual observatory network. One of the most powerful features of VxOware is the ability to link any type of object in the observatory to other objects and the ability for every object to be tagged.
Paper in Earth Science Informatics: [1] | cached copy of journal article
3. The Section
From [2]
The section is the most basic grouping of metadata records in VxOware. A virtual observatory using an implementation of VxOware may contain any number of sections, each of them having different types of metadata records. Each section has an associated XML schema and may have a render XSLT file. For example, one section may contain metadata records that follow the RSS (Really Simple Syndication) document model. When a user uses a web browser selectsto view a metadata record from this section, an XSLT transform is applied to the raw XML to render it in HTML. Besides a render XSLT file, the section may have an edit XSLT file that creates an editable HTML form. Figure 12a showis a screenshot in which athat shows a previously stored metadata record that has been rendered in HTML using a render XSLT file that is associated with that section. Figure 12b shows the rendering of the same metadata record using an edit XSLT file. The XSLT file uses the raw metadata to create links to other services. For example, the Google and Yahoo links do a search on the person’s name, the NASA/ADS link does a search for journal and conference abstracts in NASA’s Abstract Data Service, and the address link does a search on the address in Google Maps.
Any XML metadata schema can be used to form a section. The predefined VxOware sections include News, which is used for notifications and automatically generated messages and is also the source of the VxOware RSS feed; Blogs, which contains records for user notes;, Message Boards, which contains discussions between the users; and Documentation, which has an editor that accepts wiki-formatted text. Other schemas have been used in VxOware and have render and edit XSLT files that are included in the application installation package available. These schemas include SPASE (Space Physics Archive Search and Extract) [3], FGDC (Federal Geographic Data Committee) [4], WISER (Wireless Information System for Emergency Responders) [5], and OE (Ordering Extensions) [6].
4. Search: Internal
4.1. Free Search
Free text search (left box on main page and text box in a given section) uses eXist full-text search on all sections having an associated special file in eXist settings folder /system/config/db/path/to/section/collection.xconf. By default, all sections have this applied because the default database you install in eXist when VxOware is installed has the file /system/config/db/myVxO/collection.xconf in the root of the VxO tree. (You may want to change this if size of db gets large and memory is an issue). For more about full-text search configuration, see eXist documentation [7].
Pattern matching is allowed (also in free-text and advanced search, but not "outer search"). For example, searching on a time as in 2008-01* will result in all documents with that year and month as the value for any element in the record (and the forum record if SQL search is used).
Both of the operators &= and |= accept simple wildcards in the keyword string. A ? matches zero or one character, * matches zero or more characters. You may use a backslash to escape wildcard characters.
Example: To find "havana" you can use:
- havana
- ha*na
- ha?a?a
- ha*
When you do a section-specific search, either eXist or SQL is used depending on if IS_FAST_INDEX is set in vo_description.xml (See Configuration). When a section is created with IS_FAST_INDEX set, any documents that are created are stored in eXist are also copied into a SQL database and the search is performed on the SQL database contents. Note that when IS_FAST_INDEX is set in vo_description.xml for a section, the admin must also create the SQL database in the SQL Index section of the Admin management page.
There are a few small differences between SQL and eXist search (see also #Technical_Details):
- In SQL mode, the data that are indexed include information from the forum file (such as upload date, etc.; See the SQL browser in the admin section for more information). To see the difference, do a search on a date when a file was uploaded with and without
IS_FAST_INDEXset to true. - In SQL mode, the search has a slightly different notation, for example, the + and - characters at the beginning and end are interpreted differently [8]. These characters may be escaped by double quoting the entire search string.
- In SQL mode, space-separated words are connected by OR; in XML they are connected by AND.
4.2. Attribute Search
A IS_FAST_INDEX section can have special attributes associated with each record. A connection can be made between an element in the schema that can be used to mean FROM and TO coverage dates. This is indicated by placing
<IS_DISPLAY_DATE_TO/>
in the appropriate field in the schema file for that section. See the section "CoverageDemo". Other options include:
<IS_DISPLAY_N_COVERAGE/> <IS_DISPLAY_E_COVERAGE/> <IS_DISPLAY_S_COVERAGE/> <IS_DISPLAY_W_COVERAGE/> <IS_DISPLAY_DATE_FROM/>
When searching a section with any of these special attributes, an entry box will appear for search on either time or space. The spatial search uses
SELECT count(*) FROM `test_f` INNER JOIN `test_p` ON `test_f`.pathId = `test_p`.id AND `documentPath` LIKE '/Test/%' AND (contains(`spcover`, GeomFromText('POINT(0 0)')))
If one document does not have temporal coverage information, the value of W=-180,S=-90,E=180,N=90 is assumed. If no start time is given, only the stop time is used for a constraint. If no stop time is given, only the start time is used for a constraint. All documents with no start or stop time will match any constraint.
4.3. Technical Details
Example: Debug output from SQL search on 2010 (using Person section):
SELECT count(*) FROM `spaseperson_f` INNER JOIN `spaseperson_p` ON `spaseperson_f`.pathId = `spaseperson_p`.id AND `documentPath` LIKE '/SPASEPerson/%' AND (MATCH (`viewCount`) AGAINST ('2010' IN BOOLEAN MODE) OR MATCH (`replyCount`) AGAINST ('2010' IN BOOLEAN MODE) OR MATCH (`publicationDate`) AGAINST ('2010' IN BOOLEAN MODE) OR MATCH (`pathId`) AGAINST ('2010' IN BOOLEAN MODE) OR MATCH (`periodStart`) AGAINST ('2010' IN BOOLEAN MODE) OR MATCH (`periodEnd`) AGAINST ('2010' IN BOOLEAN MODE) OR MATCH (`keyVersion`) AGAINST ('2010' IN BOOLEAN MODE) OR MATCH (`keyPersonName`) AGAINST ('2010' IN BOOLEAN MODE) OR MATCH (`keyResourceID`) AGAINST ('2010' IN BOOLEAN MODE) OR MATCH (`keyReleaseDate`) AGAINST ('2010' IN BOOLEAN MODE) OR MATCH (`keyOrganizationName`) AGAINST ('2010' IN BOOLEAN MODE) OR MATCH (`keyAddress`) AGAINST ('2010' IN BOOLEAN MODE) OR MATCH (`keyEmail`) AGAINST ('2010' IN BOOLEAN MODE))
Example: Debug output from XML search on 2010 (using Wiki section):
In terms of eXist notation, the search syntax is node-set &= 'string of keywords'. This operator selects context nodes containing ALL of the keywords in the right-hand argument in any order. The default tokenizer is used to split the right-hand argument into single tokens, i.e. any punctuation or white spaces are used to separate the keywords and, after which, are omitted. Note also that wildcards are allowed, and keyword comparison is NOT case-sensitive. (There is also |= operator available, for searching ANY of the keywords. This is not used, but could be in principle.)
5. Search: External
An external search may be made on VxO sections that are available using a framework of web-services. Thus, a p2p VxO federation may be created which allows for the interaction and data exchange across different application domains. To make a section available for external search, an administrator must specify a list of sections that are web-service accessible along with a list of metadata fields that are available for searching and displaying.
The external search web-service is available via REST protocol. The web-service returns a configuration-based self-description of VxO (information about the available capabilities) with a list of sections open for search. On requesting the information for a certain section, the web service returns a list of meta-record elements (and possibly a set of valid values), with values that may be used to find objects in this section. With this self-description, a context search query can be created which results in a list of objects. An important feature of the external search web-service is its dynamic result format. You may choose which of the available meta-record fields you need to see in the query result. These properties of the web-service make it suitable for case-based reasoning applications.
Searching in a time field with, for example, 2008-01, will result in all documents with that year and month.
5.1. Configuration
5.1.1. With text editor
External search capabilities are set in the VxO configuration files.
Observatory sections can be opened for the external search by the settings of the file /conf/vo_description.xml
<?xml version="1.0" encoding="UTF-8"?> <OBSERVATORY> ... <DATA_SECTIONS> <SECTION> <NAME>Documents</NAME> <!-- sectionId --> <LONG_NAME>Documents</LONG_NAME> ... <IS_CAPABILITIES>true</IS_CAPABILITIES> </SECTION> ... </DATA_SECTIONS> </OBSERVATORY>
The list of sections available for the external search is reported in the VO capabilities document.
A list of elements exposed for the external search is set in the observatory section configuration file /conf/[sectionId]/[sectionId]Schema.xml
<?xml version="1.0" encoding="UTF-8"?> <SCHEMA> <SCHEMA_NAME>News</SCHEMA_NAME> ... <FIELD> <KEY>keyTitle</KEY> <!-- parameterId --> <LONGNAME>Title</LONGNAME> ... <IS_CAPABILITIES use="identification"/> </FIELD> .... </SCHEMA>
The list of searchable elements in a section is reported in the VO section capabilities document.
5.2. Web interface
Configuration is available on admin management page.
- External search is available only via link 1
- The list of known external sources is given in list 2
- You can add a new source in field 3
- The sources that should be available via the search pages (current VxO interface) are marked with checkboxes (4)
- The changes to the external sources list are committed by pressing button 5
- Press button 6 to automatically search for sources. When this button is clicked the system walks through all the known sources (present in the list) and makes a “shareExternalSources” request as shown below. All found sources are added to the common list.
-
- After clicking the “search source” button the known source are queried (group 1)
- The sources return the addresses of their known sources. The new addresses are added to the common list (group 2)
- VO administrator may set checkboxes for some of the new entries to make them available through the current VO interface
5.3. REST
When an xml file is uploaded into a subdirectory of an observatory section, the filename is stored internally and the xml file is given a document name that is a 36-digit hash (such as BE41D3AB-FE78-DC54-6379-2C2AB6668B96). For example, if you upload the file Robert.S.Weigel.xml, it will appear in the Exist db as virbo/SPASEPerson/BE41D3AB-FE78-DC54-6379-2C2AB6668B96.xml. You may access the xml file by filename and directory path or by its hash name. The raw xml is accessible via
- http://virbo.org/meta/viewDataFile.jsp?docname=20F920DB-C4CE-D3A1-1B51-A1BB43440DD4&filetype=data
- http://virbo.org/meta/viewDataFile.jsp?filepath=/SPASEPerson/Robert.S.Weigel.xml&filetype=data
This will also work:
Associated with each uploaded object is a forum object which includes information such as the name of the person who uploaded the object, the number times it was viewed, revision history, tags, cross-references, and discussion. To access the uploaded object in the observatory with a browser and see the HTML rendering of the uploaded metadata object and its associated forum object, use
- http://virbo.org/meta/viewdata.do?docname=BE41D3AB-FE78-DC54-6379-2C2AB6668B96
- http://virbo.org/meta/viewdata.do?filepath=/SPASEPerson/Robert.S.Weigel.xml
The raw xml for the form object is also accessible:
- http://virbo.org/meta/getobject.do?id=20F920DB-C4CE-D3A1-1B51-A1BB43440DD4&filetype=forum
- http://virbo.org/meta/getobject.do?filepath=/SPASEPerson/Robert.S.Weigel.xml&filetype=forum
5.4. Query and Result
Each parameter in self-description may be used for searching by element. Each parameter may be returned in search results
5.4.1. Query
5.4.1.1. Capabilities query
This document describes the observatory structure, its sections available for outer search
VO self-description format (http://vo_app/capabilities.do)
<?xml version="1.0" encoding="UTF-8"?> <CAPABILITIES> <NAME>VO name</NAME> <DESCRIPTION>VO description</DESCRIPTION> <SECTION> <NAME>section name</NAME> <DESCRIPTION>section description</DESCRIPTION> <PARAMKEY>sectionId</PARAMKEY> </SECTION> <SECTION>…</SECTION> </CAPABILITIES>
This document describes the records contained in sections. Each parameter describes the document’s XML element available for outer search
Section self-description format (http://vo_app/capabilities.do?section=[sectionId])
<?xml version="1.0" encoding="UTF-8"?> <CAPABILITIES> <ACTION>outersearch</ACTION> <SECTION>sectionId</SECTION> <PARAMETER> <NAME>search parameter name</NAME> <DESCRIPTION>parameter description</DESCRIPTION> <PARAMKEY>parameterId</PARAMKEY> <TYPE use=“[notes]">parameterType</TYPE> <VALUE>parameter values (optional)</VALUE> <VALUE>…</VALUE> </PARAMETER> <PARAMETER>...</PARAMETER> </CAPABILITIES>
5.4.1.2. Basic Query
http://vo_app/outersearch?section=[sectionId]&[parameterId]=[searchString]&[parameterId(2)]=[searchString(2)]&output=[parameterId(3)]&output=[parameterId(4)]
5.4.1.3. Query Option: noOutput
If «output» parameters are missing, the result will contain only the number of found records
<?xml version="1.0" encoding="UTF-8"?> <result found_items=“[int]”> <found_items>[int]</found_items> </result>
http://vo-app/outersearch?[searchRequest]&noOutput=true
This parameter can be used to obtain only the list of found documents, for example:
<?xml version="1.0" encoding="UTF-8"?> <result found_items="508"> <found_items>508</found_items> <item doc_name="GeomStationsNPL">noDetails</item> <item doc_name="GeomStationsEPN">noDetails</item> <item doc_name="GeomStationsWES">noDetails</item> ...... </result>
5.4.1.4. Query Option: document
Returns a full XML document stored in VO by its ID (applies only to records stored in observatory sections)
http://vo-app/outersearch?specialRequest=document&docId=[documentId]
5.4.1.5. Query Option: shareExternalSources
Request:
http://vo-app/outersearch?specialRequest=shareExternalSources
Result:
<?xml version="1.0" encoding="UTF-8"?> <EXTERNAL_SOURCES> <!-- the underscore before the S is only there to avoid a mediawiki rendering problem--> <_SOURCE capability="true">http://vo-app1/</_SOURCE> <_SOURCE capability="true">http://vo-app2/</_SOURCE> </EXTERNAL_SOURCES>
A list of external observatories (federation) is returned. These observatories are known by the requested one. The attribute “capability” indicates whether the external observatory can be searched by the outersearch web service at the requested one.
5.4.1.6. Query Option: idsList
http://vo_app/outersearch?section=[sectionId]&idsList=[documentId-1],[documentId-2],…,[documentId-n]&output=[parameterId-1]&output=[parameterId-2]
The “idsList” value can specify directly a list of documents IDs to be exported, rather than the document search.
This document ID filter can be used in addition to the document search filter, to search only for the documents from the idsList.
5.4.1.7. Query Option: pointSearch
http://vo_app/outersearch?section=[sectionId]&pointSearch=[latitude][longitude]&output=[parameterId-1]&output=[parameterId-2]
Search for the geographic location (latitude - longitude pair, space separated). The result will be all geolocated records, which include the point into the geolocation rectangle.
This geolocation filter can be used in addition to other document search filters.
5.4.1.8. Query Option: dateStart and dateEnd
http://vo_app/outersearch?section=[sectionId]&dateStart=[yyyy.mm.ddThh.mm.ssUTC]&dateEnd=[yyyy.mm.ddThh.mm.ssUTC]&output=[parameterId-1]&output=[parameterId-2]
Is used to search for the date range. Only documents with time stamp within the date range will be returned.
- This date range filter can be used in addition to other document search filters.
5.4.2. Result
Search results are returned as a file, where “item” corresponds to a found document and “element” corresponds to a parameter declared as “output”
Search result format
<?xml version="1.0" encoding="UTF-8"?> <result found_items=“[int]"> <found_items>[int]</found_items> <item doc_name=“recordId"> <element> <element_key>parameterId</element_key> <element_title>parameter name</element_title> <element_values> <value>parameter value</value> <value>…</value> </element_values> </element> <element>…</element> </item> <item>…</item> </result>
5.4.3. Web interface
 |
 |
 |
5.5. XSL Result Parser
5.5.1. REST Request
http://vo_app/outersearch?section=[sectionId]&[parameterId]=[searchString]&output=[parameterId-2]&useTransformer=true&transformer=http://host/file.xsl&resType=HTML
“resType” – optional parameter, if it is absent – return XML
5.5.2. Web interface
 |
 |
5.5.3. Example: KML-XML-SPIDR
This document describes how to transform XML stream into KML with external search results from the VxOware by applying an XSL file (referred by URL).
Real world example of such XSL can be found at http://dimetra.wdcb.ru/kokovin/kml-spidr-geom.xsl
Example of the VxOware “outersearch” request modified with the above XSL: http://dimetra.wdcb.ru:8080/spidrObservatory2/outersearch?section=GeomStations&dateStart=1900-01-01&dateEnd=2100-01-01&output=keyCitationTitle&output=keySpdomWest&output=keySpdomNorth&useTransformer=true&transformer=http%3A%2F%2Fdimetra.wdcb.ru%2Fkokovin%2Fkml-spidr-geom.xsl&resType=XML
We have a special web form in the VxOware to experiment with the XSL transformations of the outersearch results; when the XSL transformation will give a satisfactory result, one can copy the search request URL from the search results address line (in web browser) and use it in another application.
5.5.4. Java Result Parser
Java class to parse the search results document. (For more details see VxOware sources.)
String searchRequestUrl = “[request]”; ResultStructure searchResult= new ResultStructure(searchRequestUrl);
Compare Java method call with the XML(xPath) query:
result/item[itemNum]/element[element_key=“parameterId”]/element_values/value[0] searchResult.getItem(itemNum).getElement(“parameterId").getValue(0)
6. Search: eXist client
Launch eXist webstart, select binoculars and enter:
for $doc in /* [text:match-all(.,'Filename')] return concat(util:collection-name($doc), '/', util:document-name($doc))
to search for word Filename in all parts of all documents starting at the root of the eXist db. The documentation for eXist has many other examples.
7. Adding Metadata
Metadata may be added using
- Create record
- Upload record
- URL Collection
- Single XML File or ZIPed collection
- API
In all cases, the record is stored in a specified section and the docname is given a hash value. A human-readable URL may be specified in which case the filepath URL will show a regular filename for the record (given in the "Alternative URL" link").
7.1. Create Record
Using the "Create Record" button, you may create and/or edit metadata in a given section by entering information in a web-based form (login required). If the section has
<DOC_NAME_TYPE>free</DOC_NAME_TYPE>
set in vo_description.xml, then there is an option to give a name to the record that will appear as the filepath URL.
7.2. Upload Records
Using the "Upload records" button, you may upload a single XML file or a zip file with XML metadata files. See the URL Collections section for examples of URL Collection XML files. Name used in the filepath URL corresponds to the filename specified in the URL of the ingested record.
7.3. API
First, download the command-line tool from SVN: [9]
Example: modify an existing record using the VxOware API and command line tools.
Then, view existing metadata at
http://virbo.org/meta/viewdata.do?filepath=/SPASEPerson/PersonTest.xml
Download XML file
curl -o ./xml/SPASEPerson/PersonTest.xml http://aurora.gmu.edu:8080/exist/servlet/db/virbo/SPASEPerson/PersonTest.xml
Make manual edit to file at ./xml/SPASEPerson/PersonTest.xml
Upload modified file
./vxoware-upload.pl ./xml
View revised file using path ("Alternative URL")
http://virbo.org/meta/viewdata.do?filepath=/SPASEPerson/PersonTest.xml
View revised file using the document ID.
http://virbo.org/meta/viewdata.do?docname=PersonTest.xml
8. Creating a Section
8.1. Clone
You can clone a section by using the Admin management page. If the section is SQL-indexed, you must execute "Create index" in the Admin management page. See the following for a description of the section configuration files.
8.2. Manual
Start the eXist webstart client by going starting the web start program http://vxoware.org/exist/exist.jnlp
Create a directory named conf/Help
8.2.1. The section schema
Create a document called helpSchema.xml. This file is used for generating the edit page. Note that attributes in an element are ignored. It is not possible to set up a section where attributes can be specified as an editable parameter in the edit form. However, attributes may be used in the render xsl file.
<SCHEMA> <SCHEMA_NAME>News</SCHEMA_NAME> <SCHEMA_TITLE>News addition form</SCHEMA_TITLE> <SCHEMA_DESCRIPTION>Please fill in the following fields according to the descriptions:</SCHEMA_DESCRIPTION> <FIELD> <KEY>keyTitle</KEY> <LONGNAME>Title</LONGNAME> <DESCRIPTION>The name of the search result provider.</DESCRIPTION> <NOTE>Notes: Should contain the search terms themselves. Restrictions: Can not contain HTML markup. Required</NOTE> <TYPE>text</TYPE> <IS_DISPLAY_TITLE/> <IS_CAPABILITIES use="identification"/><!-- List this field in capabilities file for external search.--> <IS_FAST_INDEX/> </FIELD> <FIELD> <KEY>keyLink</KEY> <LONGNAME>Link</LONGNAME> <DESCRIPTION>A URL to the website that providers the search results.</DESCRIPTION> <NOTE>Restrictions: Can not contain HTML markup. Optional.</NOTE> <TYPE>text</TYPE> <IS_DISPLAY_LINK/> </FIELD> <FIELD> <KEY>keyDescription</KEY> <LONGNAME>Description</LONGNAME> <DESCRIPTION>A text.</DESCRIPTION> <NOTE>Restrictions: Can not contain HTML markup. Required</NOTE> <TYPE>textarea</TYPE> <IS_DISPLAY_DESCRIPTION/> <IS_CAPABILITIES use="identification"/> <IS_FAST_INDEX/> </FIELD> </SCHEMA>
8.2.2. The section template
The template is used by the section XSL file to address information. Create a document named helpTemplate.xml with the following contents:
<NewsDescription> <Title>keyTitle</Title> <Link>keyLink</Link> <Description>keyDescription</Description> </NewsDescription>
A more advanced template showing all options
<FIELD> <KEY>keyElementName</KEY><!-- This key is used to connect this element to the times in the section template file --> <LONGNAME>Long name of element</LONGNAME> <DESCRIPTION>A short description of the element that will appear in the editor</DESCRIPTION> <NOTE>A short note about valid entries for the element</NOTE> <TYPE>text</TYPE><!-- The type of form to be used in the element --> <TYPE>textarea</TYPE> <TYPE>select</TYPE><!-- A select element may be used --> <OPTION value="Asteroid"/> <OPTION value="Comet"/> <OPTION value="Earth"/> <TYPE>multipleselect</TYPE><!-- Checkbox --> <OPTION value="Asteroid"/> <OPTION value="Comet"/> <DATATEMPLATE/><!-- Default value to place in text or textarea element --> <IS_CAPABILITIES><!-- Show as searchable field in outer search --> <IS_CAPABILITIES use="identification"/><!--? --> <IS_FAST_INDEX/> <!-- If set to true, uses SQL as indexing engine for search on this element. If false, and SQL indexing is used, this element is ignored. Also, this setting is ignored if IS_FAST_INDEX is false in vo_description.xml--> <IS_DISPLAY_TITLE/><!-- Set this to true to have the value of the field be represented as the title --> <IS_DISPLAY_DESCRIPTION/><!-- Set this to true to have the value of the field be represented as the title --> <IS_DISPLAY_LINK/><!-- The link to show in the listing of each records in the section --> <IS_DISPLAY_PREVIEW_IMG/><!-- Set this to a URL to any local or remote image (not cached locally!) --> <IS_DISPLAY_N_COVERAGE default="90"/><!-- If this field represents a northern boundary --> <IS_DISPLAY_E_COVERAGE default="180"/><!-- If this field represents a northern boundary --> <IS_DISPLAY_S_COVERAGE default="-90"/> <IS_DISPLAY_W_COVERAGE default="180"/> <IS_DISPLAY_DATE_FROM/><!-- If this field represents a coverage start time. Format is any that MySQL DAYTIME supports --> <IS_DISPLAY_DATE_TO/><!-- If this field represents a coverage stop time. Format is is any that MySQL DAYTIME supports --> </FIELD>
8.2.3. The section tree
Create a document called helpTree.xml:
<SECTION> <ID>Help</ID> <!--Path of this section in eXist and top-level tree folder name --> <NAME>All Help Documents</NAME><!-- Name that will appear in the left menu..--> <NODE> <ID>AddingMetadata</ID> <!--Path of this subdir in eXist to search when this link is selected--> <URI>AddMeta</URI> <!--Path of this section when filepath= is used --> <NAME>Info about adding metadata</NAME> <FILTER_DEFAULT>false</FILTER_DEFAULT> </NODE> <NODE> <ID>gen</ID> <URI>General</URI> <NAME>General help documents</NAME> <FILTER_DEFAULT>false</FILTER_DEFAULT> </NODE> <FILTERS> <LABEL>Queries</LABEL> <!-- Top-level tree folder name --> <FILTERS> <LABEL>Magnetosphere</LABEL> <FILTERS> <LABEL>All magnetosphere satellites</LABEL> <VALUE>Earth.Magnetosphere*</VALUE> <PATH>/Observatory/Location/ObservatoryRegion</PATH> </FILTERS> <FILTERS> <LABEL>Magnetotail</LABEL> <VALUE>Earth.Magnetosphere.Magnetotail</VALUE> <PATH>/Observatory/Location/ObservatoryRegion</PATH> </FILTERS> </FILTERS> </SECTION>
8.2.4. The section XSL
<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="1.0"> <xsl:output indent="yes" method="html"/> <xsl:template match="/"> <p> </p> <p> <xsl:copy> <xsl:apply-templates select="NewsDescription/Description/text()"/> </xsl:copy> </p> <a href="{NewsDescription/Link}" target="_blank"> <xsl:value-of select="NewsDescription/Link"/> </a> <br/> </xsl:template> <xsl:template match="text()"> <xsl:call-template name="break"/> </xsl:template> <xsl:template name="break"> <xsl:param name="text" select="."/> <xsl:choose> <xsl:when test="contains($text, ' ')"> <xsl:value-of select="substring-before($text, ' ')"/> <br/> <xsl:call-template name="break"> <xsl:with-param name="text" select="substring-after($text,' ')"/> </xsl:call-template> </xsl:when> <xsl:otherwise> <xsl:value-of select="$text"/> </xsl:otherwise> </xsl:choose> </xsl:template> </xsl:stylesheet>
8.2.5. Edit vo_description.xml
<SECTION> <NAME>News</NAME><!-- Must match value of SCHEMA_NAME in schema file. This will be the name used in the URL.--> <LONG_NAME>News</LONG_NAME><!--This name appears in the section listing in the middle column when entering the observatory. Usually should be chosen to match value in tree file.--> <PATH>/</PATH> <DESCRIPTION>Any registered user may add news items.</DESCRIPTION> <TREE>News/newsTree.xml</TREE> <SCHEMA>News/newsSchema.xml</SCHEMA> <TEMPLATE>News/newsTemplate.xml</TEMPLATE> <DATAXSL>News/newsView.xsl</DATAXSL> <ADDITION_RESTRICTION>false</ADDITION_RESTRICTION> <!-- ADDITION_RESTRICTION :true|false|moderate --> <TOTAL_FILES>4</TOTAL_FILES> <LAST_ITEM>7072E59A-4BDE-339C-1D86-80416B66AEF1</LAST_ITEM> <IS_CAPABILITIES>true</IS_CAPABILITIES> <IS_FAST_INDEX>false</IS_FAST_INDEX> <DOC_NAME_TYPE>default</DOC_NAME_TYPE> <!-- DOC_NAME_TYPE : free|default --> <DEFAULT_SORT>pubdate</DEFAULT_SORT> <VISIBILITY>true</VISIBILITY> <CATALOGUE_VIEW> <STATISTICS>true</STATISTICS> <SORTING>false</SORTING> <ALPHABET>false</ALPHABET> <DESCRIPTION>true</DESCRIPTION> </CATALOGUE_VIEW> </SECTION>
8.2.6. Create Index
If the section is SQL-indexed (<IS_FAST_INDEX/>) is in the section template, you must execute "Create index" in the Admin management page.
9. Linking and Tagging
10. Viewers
When an object is placed into the observatory, it may have a viewer associated with it.
- Image section template should have. When this file is uploaded, the src/vops/storeAction/ImageStoreAction.java places file in
imageStoreDiras specified below. It also sets theOBJECT_TYPEtoimage.
<FIELD> <KEY>keyUploadFile</KEY> <LONGNAME>Upload File</LONGNAME> <DESCRIPTION>Upload Your Image</DESCRIPTION> <TYPE action="imageAction">file</TYPE> <DATATEMPLATE/> <IS_DISPLAY_FILE/> </FIELD>
- conf/vo_store.conf
# imagestore (ImageStoreAction) imageStoreDir=/var/www/voData/imageStore imageAction.exportDir=/var/www/voData/imageStore imageAction.storeClass=vobs.storeAction.ImageStoreAction imageAction.returner=imagestore?objId= imageAction.returnerPreview=imagestore?objId=
- conf/vo_store.conf
dataType.image.viewer=viewers/imageViewer.jsp
- web/dataviewer.jsp
String viewerPage = Settings.get("vo_store.dataType."+outputObjectType+".viewer");
11. Plug-ins
Plugins are used to place objects in the Observatory, either in the XML or Disk storage [10].
- conf/vo_store.conf
upload.data.storeClass=vobs.plugins.XmlUploadPlugin upload.dataZip.storeClass=vobs.plugins.ZipCollectionPlugin upload.urlColl.storeClass=vobs.plugins.UrlCollectionPlugin (not yet written) upload.dataCsv.storeClass=vobs.plugins.CsVToXMLPlugin
Correspond to
- vobs.plugins.XmlUploadPlugin.java
- vobs.plugins.ZipCollectionPlugin.java
- vobs.plugins.UrlCollectionPlugin.java
- vobs.plugins.CsvToXMLPlugin.java
All of these call src/webapp/UploadXmlDocument.java to put an xml file in database.
To integrate a new plug-in into observatory, you can create a jar and put in lib folder and edit vo_store.conf and will then be able to access a new upload plug-in without re-creating the war file. The upload plug-ins are specified in
- web/uploadMetadataForm2.jsp
<input type="radio" name="uploadFileType" value="data" /> <input type="radio" name="uploadFileType" value="dataZip" /> <input type="radio" name="uploadFileType" value="urlColl" /> <input type="radio" name="uploadFileType" value="dataCsv" />
12. Admin Tasks
12.1. Batch modification of records
(Using eXist Web Start interface query dialog that opens when clicking on binoculars.)
Delete note named Url on all records in SECTIONNAME.
for $url in collection("/db/virbo/SECTIONNAME")//Url return update delete $url
Replace old_string with new_string in all records:
for $doc in collection("/db/collection/")
return update replace $doc//text()["old_string"] with "new_string"
Remove ".xml" extension from OBJECT_NAME elements in forum file:
for $objname in collection("/db/virbo/forum")//DISCUSSION[ends-with(OBJECT_NAME,".xml")]//OBJECT_NAME//text()
return update replace $objname with substring-before($objname, '.xml')
To search for word Filename in all parts of all documents starting at the root of the eXist db. The documentation for eXist has many other examples.
for $doc in /* [text:match-all(.,'Filename')] return concat(util:collection-name($doc), '/', util:document-name($doc))
Copy all records from a dir to another dir. Change record name based on element in record
let $dummy1 := xmldb:authenticate('/db/system/config', 'admin', '****EDIT THIS****')
for $obj in collection('/db/virbo/PersonUnderReview')
let $fname := util:document-name($obj)
let $resid := $obj//ResourceID/text()
let $fname2 := replace($resid,"spase://SMWG/Person/","")
let $tmp := xmldb:copy("/db/virbo/PersonUnderReview","/tmp",$fname)
let $tmp2 := xmldb:rename("/tmp", $fname, concat($fname2,".xml"))
return <result>{$fname}, {$fname}, {$resid}, {concat($fname2,".xml")}</result>
Same as above but with subdirs for InstrumentUnderReview
let $dummy1 := xmldb:authenticate('/db/system/config', 'admin', '****EDIT THIS****')
for $obj in collection('/db/virbo/InstrumentUnderReview')
let $fname-uuid := util:document-name($obj)
let $resid := $obj//ResourceID/text()
let $fname-spase := concat(replace($resid,"spase://SMWG/Instrument/",""),'.xml')
let $coll-exist := xmldb:collection-available(concat("/db","tmp"))
let $tmp1 := xmldb:create-collection("/db","tmp")
let $tmp2 := xmldb:create-collection("/tmp",tokenize($fname-spase,'/')[1])
let $fname-tmp := concat("/tmp/",$fname-uuid)
let $dest-path := concat("/tmp/",tokenize($fname-spase,'/')[1])
let $tmp3 := xmldb:copy("/db/virbo/InstrumentUnderReview",$dest-path,$fname-uuid)
let $tmp4 := xmldb:rename($dest-path, $fname-uuid, tokenize($fname-spase,"/")[2])
return <result>{$coll-exist}, {$fname-uuid}, {$fname-tmp}, {$fname-spase}</result>
for PersonUnderReview section
let $dummy1 := xmldb:authenticate('/db/system/config', 'admin', '****EDIT THIS****')
for $obj in collection('/db/virbo/PersonUnderReview')
let $fname-uuid := util:document-name($obj)
let $resid := $obj//ResourceID/text()
let $fname-spase := concat(replace($resid,"spase://SMWG/Person/",""),'.xml')
let $fname-tmp := concat("/tmp/",$fname-uuid)
let $coll-exist := xmldb:collection-available(concat("/db","tmp"))
let $tmp1 := xmldb:create-collection("/db","tmp")
let $tmp2 := xmldb:create-collection("/tmp",tokenize($fname-spase,'/')[0])
let $dest-path := concat("/tmp/",tokenize($fname-spase,'/')[1])
let $tmp3 := xmldb:copy("/db/virbo/PersonUnderReview",$dest-path,$fname-uuid)
let $tmp4 := xmldb:rename($dest-path, $fname-uuid, $fname-spase)
return <result>{$coll-exist}, {$fname-uuid}, {$fname-tmp}, {$fname-spase}</result>
Copy all records with tag "toreview" to a temp directory with filename corresponding to OBJECT_NAME
let $dummy1 := xmldb:authenticate('/db/system/config', 'admin', '****EDIT THIS****')
for $objname in collection("/db/virbo/forum")//TAG [text:match-all(.,'toreview')]
let $store-return := xmldb:copy("/db/virbo/forum", "/db/tmp", util:document-name($objname) )
let $object-path := $objname/ancestor::*//OBJECT_PATH
let $coll-exist := xmldb:collection-available(concat("/db/tmp",$object-path))
let $tmp := if ($coll-exist) then $coll-exist else xmldb:create-collection("/db/tmp",$object-path)
let $filename := concat($objname/ancestor::*//OBJECT_NAME,".xml")
let $source-path := concat("/db/virbo",$object-path)
let $dest-path := concat("/db/tmp",$object-path)
let $obj-id := concat($objname/ancestor::*//OBJECT_ID,".xml")
let $tmp := xmldb:copy($source-path,$dest-path,$obj-id)
let $tmp2 := xmldb:rename($dest-path,$obj-id,$filename)
return <file>{concat($source-path, " ", $dest-path, " ", $objname/ancestor::*//OBJECT_PATH, $objname/ancestor::*//OBJECT_NAME,".xml", " ", $coll-exist, " ", util:document-name($objname))}</file>
12.2. Command line eXist
sudo ./client.sh -s -u admin --password=PASSWORD
13. The forum file
Each record has an associated "forum" file that contains metadata about the record.
<DISCUSSION> <OBJECT_ID>19F65213-17B6-237A-7664-83E8B648B664</OBJECT_ID> <AUTHOR_NAME>admin</AUTHOR_NAME> <AUTHOR_ID>CC7796C3-796F-8D39-F315-8C4803F0E1CA</AUTHOR_ID> <CO_AUTHORS/> <NOTIFY_AUTHOR>true</NOTIFY_AUTHOR> <OBJECT_SECTION>PubBlogs</OBJECT_SECTION> <OBJECT_PATH>/PubBlogs/PubBlogs/</OBJECT_PATH> <OBJECT_NAME/> <FILE_PATH/> <VIEW_COUNTER>0</VIEW_COUNTER> <BASKET_COUNTER>0</BASKET_COUNTER> <BASKET_USERS_ID/> <TITLE>Tee</TITLE> <DESCRIPTION>Sdf</DESCRIPTION> <ONLINK/> <NCOVER/> <ECOVER/> <SCOVER/> <WCOVER/> <PERIOD_START/> <PERIOD_END/> <PREVIEW_IMG/> <DOC_INDEX/> <PUBDATE>2010-01-30T20:42:17</PUBDATE> <PUB_PC_DATE>1264884137772</PUB_PC_DATE> <REPLIES discussRestriction="true"/> <VOTING averaged="0" votingRestriction="false"/> <EDITORS/> <TAGS/> <RELATED_OBJECTS/> <WORKFLOW/> </DISCUSSION>
14. Configuration
14.1. vo_description.xml
<OBSERVATORY> <NAME>myVxO</NAME> <LONG_NAME>My Virtual Observatory</LONG_NAME> <DESCRIPTION>Description of my Virtual Observatory</DESCRIPTION> <URL>http://localhost:8080/myVxO</URL> <SEARCH_URL>/</SEARCH_URL> <HEADER>html/null.html</HEADER> <MENU_FOOTER>html/menuFooter.html</MENU_FOOTER> <FOOTER>html/Footer.html</FOOTER> <LOGO_IMAGE>images/null.gif</LOGO_IMAGE> <INTRO_IMAGE>images/null.gif</INTRO_IMAGE> <INTRO> </INTRO> <MIRRORS> <MIRROR>NULL</MIRROR> </MIRRORS> <CNTINFO> <CNTADDR> <CITY>BOULDER</CITY> <COUNTRY>USA</COUNTRY> </CNTADDR> <CNTEMAIL>jjn@wdcb.ru</CNTEMAIL> </CNTINFO> <INTERFACE_LANGUAGE>english</INTERFACE_LANGUAGE> <ACTUAL_OPENING_DATE>01/01/2000</ACTUAL_OPENING_DATE> <OBSERVATORY_NEWS value=""/> <DATA_SECTIONS> <SECTION> <NAME>SPASEPerson</NAME> <!-- Where records are stored in eXist db /db/myVxO/SPASEPerson --> <LONG_NAME>Person</LONG_NAME><!-- Name that appears on left menu --> <PATH>/</PATH> <DESCRIPTION>SPASE Records for persons from SMWG repository [http://groups.google.com/group/vxo-spase-smwg]</DESCRIPTION> <TREE>SPASEPerson/spaseTree_5.xml</TREE> <!-- Location of config files (/db/myVxO/conf/SPASEPerson) --> <SCHEMA>SPASEPerson/spaseSchema_5.xml</SCHEMA> <TEMPLATE>SPASEPerson/spaseTemplate_5.xml</TEMPLATE> <DATAXSL>SPASEPerson/spase3.xsl</DATAXSL> <!-- Leave empty to display raw XML --> <PRE_DATA_URL>SPASEPerson/preprocess.xsl</PRE_DATA_URL> <!- Allow user to specify the URL to a record and apply this xsl transformation to record before putting in data base. See Ingest Record Demo section --> <EDITOR_MODE>form</EDITOR_MODE><!-- Create Record mode form|text|upload. If text, a textarea is given for entering XML. If upload, an upload form is shown. If form (default), then a text entry form is shown. --> <ADDITION_RESTRICTION>true</ADDITION_RESTRICTION><!-- true|false|moderate (true = admin only, false = all users, moderate = admin must approve using catalogue.jsp?searchSection=/waitingForApproval --> <TOTAL_FILES>4448</TOTAL_FILES> <LAST_ITEM>75BA2CB6-DECB-5B83-6F9D-6A1D0961BFD2</LAST_ITEM> <IS_CAPABILITIES>true</IS_CAPABILITIES> <IS_FAST_INDEX>true</IS_FAST_INDEX><!-- Set to true to use SQL indexing on this section. If this is changed to true after documents exist in this section, you must initiate indexing from the admin page --> <DOC_NAME_TYPE>default</DOC_NAME_TYPE><!-- DOC_NAME_TYPE : free|default if free, then allow user to specify a document name (as in the Images section) --> <DEFAULT_SORT/> <VISIBILITY>true</VISIBILITY> <PARENT>SPASE Metadata</PARENT> <CATALOGUE_VIEW> <STATISTICS>true</STATISTICS> <SORTING>false</SORTING> <ALPHABET>false</ALPHABET> <DESCRIPTION>true</DESCRIPTION> </CATALOGUE_VIEW> </SECTION>
14.2. Config files
In WEB-INF/web.xml, there is a variable that points to init.conf. This is the main configuration file. Note that Windows-style paths must be the old eight-character style (C:\PROGRA~1\etc\). To figure out the correct path, use dir \x at a MS-DOS prompt.
14.2.1. mail.conf
Configure the sender of emails from the system. When an i-mail is sent to a user, it will also be forwarded to the user's external email address. The sender of the imail will be the account specified in smtpUser.
14.3. vo_store.conf
Set the directories on the local disk where various files are stored, including images and files.
14.4. vo_meta.conf
Set information about the location of the eXist database and the path to the observatory information within the database.
14.5. textindex.conf
SQL database configuration (for sections indexed with SQL)
14.6. users.conf
SQL database for user configuration information (username, password, email, etc.)
14.7. Logging
Catalina: Edit WEB-INF/classes/log4j.properties parameter
log4j.appender.T.Directory
The parameter
logPath
in init.conf is where adminviewlog.do looks for these log files.
Other: Edit init.conf parameter (jdbc:mysql connection log)
logFile
15. Installation
15.1. General Instructions
15.1.1. Pre-requisites
- Java (http://java.sun.com/) (Set the JAVA_HOME environment variable e.g., setenv JAVA_HOME /usr/local/jdk1.5.0)
- Ant (http://ant.apache.org/) version 7 (if compiling from source)
- Apache Tomcat (http://tomcat.apache.org/) version 6
- eXist XML database (http://exist-db.org/)
15.1.2. Install and configure Java
Please refer to your Linux distribution manual about installing the Java package. The Sun JDK version is preferrable. You can obtain it from http://java.sun.com.
15.1.3. Install and configure Tomcat
The Tomcat installation directory (e.g., /usr/local/tomcat) will be referred to as CATALINA_HOME.
Enable Manager Servlet (see also http://tomcat.apache.org/tomcat-6.0-doc/manager-howto.html)
- add "manager" role to tomcat's conf/tomcat-users.xml
- add user with "manager" role
<role rolename="manager"/> <user username="tomcat" password="secret" roles="manager"/>
set privileged="true" in conf/context.xml
<Context> <Context privileged="true"/> ...
Change session length from default of 30 minutes to 180 minutes
<session-config>
<session-timeout>180</session-timeout>
</session-config>
Retart Tomcat:
$CATALINA_HOME/bin/shutdown.sh $CATALINA_HOME/bin/startup.sh
Test the Tomcat manager by pointing your web browser to:
http://localhost:8080/manager/html
15.1.4. Install MySQL
Please refer to your Linux distribution manual about installing MySQL. A useful browser interface for mySQL is http://sidu.sourceforge.net/sidu/sidu-download.php
15.1.5. eXist
Download eXist, configure it, and install into Tomcat
15.1.5.1. eXist 1.4
wget http://sourceforge.net/projects/exist/files/Stable/1.4/eXist-setup-1.4.0-rev10440.jar/download
mkdir /usr/local/eXist jar -xvf -c /usr/local/eXist eXist-setup-1.4.0-rev10440.jar
Change settings to use port 8088 in file /usr/local/bin/eXist/tools/jetty/etc/jetty.xml
<SystemProperty name="jetty.port" default="8088"/>
OR add a line to file /usr/local/eXist/tools/wrapper/conf/wrapper.conf
wrapper.java.additional.6=-Djetty.port=8088
and start Jetty with
/usr/local/bin/eXist/bin/startup.sh
OR to run as a service make link:
sudo ln -s /usr/local/eXist/tools/wrapper/bin/exist.sh /etc/init.d/exist
and start as:
sudo service exist start
To install eXist in x64 Ubuntu as a service, use the x64 wrapper from http://jira.codehaus.org/browse/MRM-923
To restore an eXist backup:
/usr/local/eXist# ./bin/backup.sh -u admin -p PASSWORD -r /usr/local/eXist/webapp/WEB-INF/data/export/fulldate-time.zip --option uri=xmldb:exist://localhost:8086/exist/xmlrpc
15.1.5.2. eXist 1.2
(VxOware past version 20100801 requires eXist 1.4. Previous versions may use eXist 1.2 or 1.4.)
Download the eXist war file from http://exist-db.org/download.html
wget http://prdownloads.sourceforge.net/exist/eXist-1.2.2-rev7828.war
Copy the eXist war file into Tomcat's webapps directory as "exist.war"
cp eXist-1.2.2-rev7828.war $CATALINA_HOME/webapps/exist.war
Test the eXist install by opening http://localhost:8080/exist/
15.1.5.3. Common to 1.2/1.4
Run the eXist Webstart client via the "Launch" button near the lower left of your eXist web page. Login as admin. You do not need a password the first time you login.
To set the admin password, select
- Tools/Edit Users
- Select "admin" in the list.
- Enter a password.
Click "Modify User" (which provides very little apparent confirmation as a result)
Add the following to the <scheduler> section of conf.xml (which is located in the eXist installation directory). The backups will be located in
/usr/local/eXist/webapp/WEB-INF/data/export
<job type="system" name="check1" class="org.exist.storage.ConsistencyCheckTask" cron-trigger="0 0 * * * ?"> <parameter name="output" value="export"/> <parameter name="backup" value="yes"/> <parameter name="incremental" value="no"/> <parameter name="incremental-check" value="no"/> <parameter name="max" value="2"/> </job> <job type="system" name="check2" class="org.exist.storage.ConsistencyCheckTask" period="7200000"> <parameter name="output" value="export"/> <parameter name="backup" value="yes"/> <parameter name="incremental" value="yes"/> <parameter name="incremental-check" value="no"/> </job>
15.1.6. Install
You can install VxOware from source by checking it out and building from SVN or by downloading a zip file that contains the war archive along with default databases and config files.
SVN-specific instructions are shown in red, zip file instructions in yellow.
* Download the zip file using the "Download Now!" link at https://sourceforge.net/projects/vxoware * Unzip the file and rename the directory vxoware * move the war file to the tomcat webapps directory ($CATALINA_HOME/webapps)
Get the application from svn svn co https://vxoware.svn.sourceforge.net/svnroot/vxoware/trunk vxoware In build.properties, * set catalina.home to $CATALINA_HOME * set username and password to that of tomcat manager * set the app.name property to myVxO * set manager.username property to define the name of a user having the manager role Create a link to catalina-ant task in your home directory. mkdir -p .ant/lib ln -s $CATALINA_HOME/lib/catalina-ant.jar ~/.ant/lib
Download Ant from http://ant.apache.org/bindownload.cgi. You may also need to add the ant bin folder to the path. Build and deploy the web application ant install If the webapp deployment fails because it already exists, try: ant try
Open http://localhost:8080/myVxO The default users and their passwords already existing in installation are
admin:admin user1:user1 user2:user2
15.1.7. Set default databases
SVN-specific instructions are shown in red, zip file instructions in yellow.
Load vxoware/database/virtobs.sql into MySQL
mysql -u login -p -h localhost < virtobs.sql
Unzip voData directory
unzip vxoware/databse/voData.zip
Move to appropriate directory
mv vxoware/database/voData $SOMEDIR
and edit configuration file accordingly
vxoware/conf/vo_store.conf
Unzip vxoware/database/myVxO.zip
cd vxoware/database; unzip myVxO.zip
and insert into eXist:
* Open http://localhost:8080/exist * Press orange Launch button near the bottom of the left column * Enter the password you entered installing eXist * Press OK * Select tools -> Restore * Files of type: __contents__.xml files
Browse to the vxoware/myVxO/db and press 'Select backup file for restore'
15.1.8. Move config files
Set up a configuration directory, e.g.,
mkdir /etc/myVxO ; cp -r vxoware/conf /etc/myVxO/conf
Edit the file /etc/myVxO/conf/init.conf
set directory.conf=/etc/myVxO/conf
Edit other locations in settings file
In vo_meta.conf (e.g., /etc/myVxO/conf/vo_meta.conf), set the eXist database admin password.
You may also set the language and time format here.
15.1.9. Modify log4j.properties
Edit this line in vxoware/web/WEB-INF/classes
log4j.appender.T.Directory
it must be a directory path and not a file path, e.g.,
/tmp
15.1.10. Modify web.xml
In $CATILINA_HOME/webapps/vxoware/WEB-INF/web.xml, change the SettingsServlet configFile element to define the path of the permanent configuration file init.conf, e.g.,
<servlet> <servlet-name>SettingsServlet</servlet-name> <servlet-class>wdc.settings.SettingsServlet</servlet-class> <init-param> <param-name>configFile</param-name> <param-value>/etc/myVxO/conf/init.conf</param-value> </init-param> <load-on-startup>1</load-on-startup> </servlet>
15.1.11. Apache config
Options Indexes FollowSymLinks MultiViews <Proxy *> Order Allow,Deny Allow from all </Proxy> ProxyPass /myVxO ajp://localhost:8009/myVxO ProxyPassReverse /myVxO ajp://localhost:8009/myVxO
To change name, place this before above
RewriteEngine On RewriteRule ^/NewName/(.*)$ /myVxO/$1 [PT,L,QSA]
15.2. Eclipse
Note that we have successfully built and deployed VxOware in Tomcat using Eclipse Java EE IDE for Web Developers (Eclipse build id: 20100218-1602). (eXist and SQL were not running in Eclipse, however; they were run on a remote server.) We used an external eXist and SQL database.
Tested on Eclipse-jee-Europa-Winter and others
- Check-out from SVN
- Install Tomcat 6
- Edit location of Tomcat in build.properties
- Edit location of init.conf in web/WEB-INF/web.xml
- Project->Properties->Java Build Path->Libraries tab should be the same JDK (1.5+, and not a JRE) as Window->Preferences->Java->Installed JREs
- "Windows -> Preferences -> Ant -> Runtime, click on the Ant Home Entries (Default), and add your required JARs by clicking on the Add External JARs. The JAR file that contains Tomcat's Ant tasks is located %TOMCAT_HOME%\server\lib and is named catalina-ant.jar. By adding it to the Ant's runtime, you can now define Tomcat tasks in the Ant editor and invoke them if needed." [11]
- Edit conf/*.conf paths
- Select Tomcat under Servers tab -> Right-click -> Add and Remove -> Remove all is sometimes needed to get to run in Eclipse Galileo if 404 appears (some files are not being copied to local Tomcat webapp dir under Eclipse?).
To connect to ec2 user db, use (USERNAME and PASSWORD are in users.conf)
mysql -u USERNAME -pPASSWORD virtobs grant all on virtobs.* to 'USERNAME'@'IP' identified by 'PASSWORD'; flush privileges;
To connect to ec2 user SQL index (used by some sections), use (USERNAME and PASSWORD are in textindex.conf)
mysql -u USERNAME -pPASSWORD textindex grant all on textindex.* to 'USERNAME'@'IP' identified by 'PASSWORD'; flush privileges;
15.2.1. eXist Plugin
Use
- Install URI: http://exist.sourceforge.net/plugin-update-site/
- Uncheck Group items by Category (Ohterwise, on Galileo, nothing will be listed for install.)
15.3. EC2 Instructions
First, obtain an EC2 account and install the ec2 tools on your local machine by following https://help.ubuntu.com/community/EC2StartersGuide
From command line, execute
ec2-run-instances ami-bb709dd2 -k ec2-keypair
Wait a minute or two until
ec2-describe-instances
returns with a URL to your instance such as ec2-174-129-50-195.compute-1.amazonaws.com
ec2-authorize default -p 22 # Authorize SSH ec2-authorize default -p 8080 # Authorize Tomcat ec2-authorize default -p 3306 # Authorize mySQL ec2-authorize default -p 80 # Authorize mySQL ssh -i ~/.ec2/ec2-keypair.pem ubuntu@ec2-174-129-50-195.compute-1.amazonaws.com
remove * in URL http://us*.archive in sources.list
sudo nano /etc/apt/sources.list sudo apt-get update sudo tasksel --section server # Select Tomcat and LAMP. Write down mySQL password that you entered wget http://localhost:8080/ # Should work sudo apt-get install ant sudo apt-get install subversion
get jdk-6u18-linux-i586.bin from java.sun.com
sh jdk-6u18-linux-i586.bin ln -s jdk1.6.0_18 jdk JAVA_HOME=/home/ubuntu/jdk;export JAVA_HOME JRE_HOME=/home/ubuntu/jdk;export JRE_HOME CATALINA_HOME=/usr/share/tomcat6;export CATALINA_HOME
Change line
<user username="tomcat" password="tomcat" roles="tomcat"/>
to
<user username="tomcat" password="tomcat" roles="tomcat,manager"/> sudo nano /etc/tomcat6/tomcat-users.xml sudo nano /etc/tomcat6/context.xml # Add <Context privileged="true"/> inside of existing <Context> block.
Next three steps are required if using Sun's Java, which is recommended
sudo ln -s /etc/tomcat6 /usr/share/tomcat6/conf sudo ln -s /usr/share/tomcat6/webapps/default_root /usr/share/tomcat6/webapps/ROOT sudo mkdir /usr/share/tomcat6/logs
Install eXist:
wget http://sourceforge.net/projects/exist/files/Stable/1.2/eXist-1.2.6-rev9165.war/download sudo cp eXist-1.2.6-rev9165.war $CATALINA_HOME/webapps/exist.war
Re-start Tomcat:
sudo $CATALINA_HOME/bin/shutdown.sh sudo JRE_HOME=$JRE_HOME $CATALINA_HOME/bin/startup.sh
Check http://ec2-174-129-50-195.compute-1.amazonaws.com/manager/html to make sure everything is running (username and password default is tomcat)
svn co https://vxoware.svn.sourceforge.net/svnroot/vxoware/virbo/trunk vxoware sudo less /etc/tomcat6/tomcat-users.xml nano vxoware/build.properties # Set catalina.home=/usr/share/tomcat6 and change username and password to match what is in tomcat-users.xml nano vxoware/build.xml # Replace virbo with myVxO nano vxoware/web/WEB-INF/web.xml # Set SettingsServlet to /etc/myVxO/conf/init.conf nano vxoware/conf/init.conf # set directory.conf=/etc/myVxO/conf sudo mkdir /etc/myVxO sudo cp -r vxoware/conf /etc/myVxO sudo mysql -u root -p -h localhost # Enter create database myVxO;quit; sudo mysql -u root -p -h localhost myVxO < vxoware/user_sql_database/virtobs.sql sudo apt-get install unzip cd /var/www; sudo unzip ~/vxoware/voData/voData.zip scp vxoware/exist_database/backup_virbo.zip yourlocalmachine: unzip backup_virbo.zip Run eXist webstart client from http://ec2-174-129-50-195.compute-1.amazonaws.com:8080/exist Select tools -> Restore Files of type: __contents__.xml files
Browse to the vxoware/exist_database/db and press 'Select backup file for restore' and select __contents__.xml file
sudo ln -s /usr/share/tomcat6 /usr/local/tomcat mkdir -p ~/.ant/lib ln -s $CATALINA_HOME/lib/catalina-ant.jar ~/.ant/lib
Note that sudo ant try does not pass user's environment vars
for JAVA_HOME and CATALINA_HOME, so sudo cp is needed.
cd vxoware ; ant try sudo cp vxoware/deploy/myVxO.war /usr/share/tomcat6/webapps
