Upgrading to dotCMS 5.0

dotCMS 5.0 is a major upgrade which includes changes to core dotCMS functionality such as Workflows and core dotCMS components including Elasticsearch. Due to the nature of these changes, it is strongly recommended that you read this entire document before upgrading to dotCMS 5.0 from an earlier version of dotCMS, and perform the steps recommended in this document during your upgrade process to ensure that your site runs correctly after the upgrade.

Required: Full Cache Flush and Reindex

Due to the change in the Elasticsearch version used by dotCMS in version 5.0, you must perform a full reindex of your dotCMS content after upgrading to dotCMS 5.0, and perform a full reindex of all Site Search indexes. Because the Elasticsearch change affects all indexes, there is no way to reindex only portions of your dotCMS content.

It is also recommended that you perform a full cache flush before performing a reindex, to ensure that all content in the cache is replaced after the index is completed.

Required: Update Elasticsearch Queries

dotCMS 5.0 includes an upgrade of Elasticsearch from version 1.7.3 to version 6.1.3. This update includes several major changes to Elasticsearch which will prevent some queries that worked in dotCMS 4.x and below versions from working in dotCMS 5.0.

Update Queries Which Contain the _all Keyword

Elasticsearch no longer supports the use of the _all keyword in queries. In order to support existing customers and continue providing the benefits of having a “catch-all” field designator of this type, dotCMS has created a new term catchall which duplicates the functionality of the previous _all keyword.

In order to ensure your Elasticsearch queries perform the same as they did in previous dotCMS versions, you must modify all queries that use the _all keyword, replacing it with catchall.

To find and update your queries:

1. Perform the following search using the Query Tool:
   • Lucene Query: +catchall:*_all*
   • Offset: 0
   • Limit: 1000
   • Press the SUBMIT button
2. Open each content returned from this query.
3. In each content, search and replace each instance of “_all” with “catchall”.

Update Wildcard Queries for Numeric and Date/Time Fields

In earlier versions of Elasticsearch, all fields were indexed as strings. In the new version of Elasticsearch, all fields are now indexed as native types (such as integer or date), which both improves performance and enables some additional query methods with full Elasticsearch JSON queries. This includes fields of the following types:

However wildcard queries only work on strings, and this means that these field types can not be accessed using wildcards. For example, the following date query, which worked in dotCMS 4.x and earlier, no longer works in dotCMS 5.0:

+publishDate:[2015* TO 2016*]

To resolve this, you must update any queries which use wildcards to access any of the above types of fields. There are two ways you can rewrite wildcard queries for these field types:

1. For the best performance it is recommended that, whenever possible, you rewrite the query to no longer use wildcards.
2. If it is not possible or desireable to remove the wildcards from the query, you may retain the behavior of the previous query by using the raw version of the numeric or date/time field in the query.

For example, the above query can be rewritten in either of the following ways:

1. +publishDate:[20150101 TO 20161231]
2. +publishDate_dotraw:[2015* TO 2016*]

Update All Queries Which Use the yyyyMMdd date format

Elasticsearch 6.1 no longer supports the date format yyyyMMdd without the time included. Therefore all queries which use the old yyyyMMdd date format must be updated to one of the date formats supported by Elasticsearch 6.1. The following date formats are supported:

• Date-only Formats:
  • MM/dd/yyyy
• Date and Time Formats:
  • MM/dd/yyyy hh:mm:ss
  • MM/dd/yyyy hh:mm:ss [AM|PM|am|pm]
  • yyyyMMddHHmmss

Note that the simplest way to modify existing yyyyMMdd formatted dates is to add the time to the end of the date. For example the date 20170101 could be modified to 20170101000000. Please recognize that when modifying dates within a date range query (e.g. [20170101 TO 20171231]), the second date in the range should specify the last second of the day instead of the first (e.g. [20170101000000 TO 20171231235959]).

Review All Queries Which Use an Implicit “OR” (Space Between Terms)

Due to changes in Elasticsearch, queries which use a space as an “OR” between terms matching the same field may need to be modified to work properly.

To ensure these queries work properly, it is recommended that you review your queries that match multiple values for the same field, and ensure that one of the following is true:

• The query uses an explicit “OR” operator instead of an implicit OR (via a space).
  Example: +field:(term1 OR term2)
• Each term in the query explicitly specifies the field.
  Example: +(field:term1 field:term2)
• All search terms are enclosed in quotes.
  Example: +field:("term1" "term2")

Recommended: Browser Cache Flush

It is generally recommended that, after you upgrade to dotCMS 5.x, your back-end users flush their browser cache. This helps ensure that back-end User Interface elements such as Javascript, Angular components, and images from the older version do not persist and cause unexpected behavior with the new back-end interface.

Recommended: Upgrade Legacy Relationships

The new Relationships fields in dotCMS 5.0 provide increased functionality and flexibility, including the ability to have Relationships which are visible from only one side of the Relationship, the ability to have One-to-One cardinality, and the ability to query related content using the field name (rather than a separate Relationship name which compounds the names of the Content Types).

Legacy Relationships are still supported in dotCMS 5.0, but it is recommended that you review and consider migrating your existing (Legacy) Relationships to the new Relationship fields when you are able to do so.

Migrating Legacy Relationships

When you migrate a legacy Relationship, a new Relationship field is added to each of the Content Types in the original Relationship, and the cardinality of the new Relationship fields are set to match the cardinality of the original Relationship. Existing relations between individual content items are duplicated in the new Relationship fields to match the relations in the Legacy Relationship.

To migrate a Legacy Relationship to a new Relationship field:

1. Open the Content Types (Legacy) Tool in the back-end.
2. Find one of the Content Types that are part of the Relationship in the list, and click the View link in the Relationships column (at the far right of the list).
   • A list displays all Legacy Relationships the Content Type is a part of.
3. Click the Relationship Name of the Relationship you wish to migrate.
4. In the Relationship editing screen, press the CONVERT TO RELATIONSHIP FIELD button.
   • A popup will appear with a warning, asking you to confirm the migration.
5. Press the OK button to continue with the migration.
6. Important: Update all your Velocity content pulls to pull from the new Relationship fields instead of the Legacy Relationship.
   • Please see below for more information.

Migrating Legacy Relationship Code

After migrating a Legacy Relationship to new Relationship fields, you must modify any Velocity code which pulls related content to replace the old $dotcontent.pullRelated() method with the new $dotcontent.pullRelatedField() method.

For more information on the pullRelated and pullRelatedField content pull methods, please see the Pull and Display Related Content documentation.

Upgrading from Earlier Releases

You can upgrade to dotCMS 5.x from releases earlier than dotCMS 4.x. In most cases the upgrade may be performed directly, and dotCMS will upgrade properly. However in some cases you may need to take additional steps to prepare your data for the upgrade; these are the same steps that would be needed to upgrade from the earlier release to dotCMS 4.x. For information on preparing an earlier release for upgrade to dotCMS 4.x, please see the Upgrading to dotCMS 4.0 documentation.