Comprehensive Guide to TYPO3 Solr Search : Series 3

Welcome to Search Series 3! Are you looking for an Apache TYPO3 Solr extension? This blog will help you from Beginner to Advanced level guide to apache’s solr TYPO3 solution.

Comprehensive Guide to TYPO3 Solr Search : Series 3

Welcome to Search Series 3! Are you looking for an Apache TYPO3 solr extension? This blog will help you from Beginner to Advanced level guide to apache’s solr TYPO3 solution.

Apache solr TYPO3 is the last blog in the TYPO3 search series, ending with my preferred search solution TYPO3 solr extension search. Compared to EXT:solr, other TYPO3 search extensions like EXT:indexed_search & EXT:ke_search, they are just like a black box. TYPO3 solr is the only search solution that smartly gives results with perfect indexing and priority to your site visitors.

TYPO3 Search Series

According to our last #T3POLL on “Best TYPO3 search solution for SMB”, Most people voted for Apache TYPO3 solr extension.

Apache solr (Searching On Lucene) is a free, open-source search engine based on the Apache Lucene library.

solr is a standalone enterprise search server with a REST-like API.

You put documents in it (called "indexing") via JSON, XML, CSV or binary over HTTP. You query it via HTTP GET and receive JSON, XML, CSV or binary results.

solr vs ElasticSearch

 

Apache solr

ElasticSearch

Nature

Open-source

Not fully open-sourced

Format

XML, CSV, JSON

Only JSON

Splitting Shards

Possible

Impossible

Documentation

Well-documented

Badly documented

Features of Apache TYPO3 solr

  • Advanced Full-Text Search Capabilities
  • Optimized for High Volume Traffic
  • Standards-Based Open Interfaces - XML, JSON and HTTP
  • Comprehensive Administration Interfaces
  • Easy Monitoring
  • Highly Scalable and Fault-Tolerant
  • Flexible and Adaptable with easy configuration
  • Near Real-Time Indexing
  • Extensible Plugin Architecture

Architecture of Apache solr

In this article, I’m going to explain, One of the most popular TYPO3 extensions called EXT:solr.

What is the architecture of Apache solr TYPO3 Extension & How it works?

What are the ways to Use Apache solr TYPO3?

The good thing about Apache solr TYPO3, You can install, configure, and use it in different ways, based on your budget and needs as follows.

Option #1 Self-Hosted Apache solr TYPO3

Both Apache solr and TYPO3 are OpenSource products, so you set it at your self-hosted or cloud-based.

Option #2 Get from Web Hosters

If you are not a technical guy, then you can use take setup it through web hosters, Jweiland, one of the oldest TYPO3 hosting provides such TYPO3 solr Web Hosting.

Option #3 Specialised Apache solr Service Provider

If you are looking for expertise in Apache solr, then connect with the team DKD who developed and maintain Apache solr TYPO3 extension.

Introduction to TYPO3 solr Extension (aka EXT:solr)

Apache solr for TYPO3 is the enterprise search server you were looking for with unique features such as Faceted Search or Synonym Support and incredibly fast response times of results within milliseconds.

solr

Extension Key

500000+

Downloads

Features of solr TYPO3 Extension

  • Autosuggest
  • Boosting of results
  • Consideration of user access rights
  • Content elevation
  • Content indexing
  • Faceted search
  • Language analysis
  • Multi-language handling
  • Performance
  • Search word highlighting
  • Sorting of results
  • Spell checker
  • Stopword support
  • Synonym support

How to Use & Install Apache solr TYPO3?

You can install and configure Apache solr with or without Docker.

Option #1 Install-solr.sh

With the extension, it ships and installs a script that can be used for a development context.

chmod u+x ./Resources/Private/Install/install-solr.sh
./Resources/Private/Install/install-solr.sh -d /home/developer

Option #2 Using Docker

You can easily install Apache TYPO3 solr extension through this docker image  hub.docker.com/r/typo3solr/ext-solr/.

// Pull TYPO3 Apache solr Docker Image
docker pull typo3solr/ext-solr:<EXT:Solr_Version>

// Start container with anonymous volume
docker run --name=typo3s-solr-server -d -p 8983:8983 typo3solr/ext-solr
docker inspect -f '{{ .Mounts }}' typo3s-solr-server

// Start container with volume on hosts path
mkdir .solrdata
docker volume create --name typo3s-solr-server-data --opt type=none --opt device=$PWD/.solrdata --opt o=bind
docker run --name=typo3s-solr-server --mount source=typo3s-solr-server-data,target=/var/solr -d -p 8983:8983 typo3solr/ext-solr

Test-Dive To Check solr Up & Runs

Deployment of EXT:solr configuration into Apache solr

All you need to do is, you need to copy the configSet directory into your prepared solr installation and replace the solr.xml file. In the installer we do it like this:

// Copy default solr file
cp -r ${EXTENSION_ROOTPATH}/Resources/Private/solr/configsets ${SOLR_INSTALL_DIR}/server/solr
cp ${EXTENSION_ROOTPATH}/Resources/Private/solr/solr.xml ${SOLR_INSTALL_DIR}/server/solr/solr.xml

// Copy the default cores:
cp -r ${EXTENSION_ROOTPATH}/Resources/Private/solr/cores ${SOLR_INSTALL_DIR}/server/solr

// Create a core with the rest API:
curl "http://localhost:8983/solr/admin/cores?action=CREATE&name=core_de&configSet=ext_solr_8_0_0&schema=german/schema.xml&dataDir=../../data/german"

Option #1 Install from TER

Go to the Extension Manager, select Get Extensions and search for “solr”.

Option #2 Install from TYPO3 solr Github

git clone git@github.com:TYPO3-solr/ext-solr.git solr

Option #3 Install with Composer

composer require apache-solr-for-typo3/solr

Let’s quickly configure the TYPO3 solr extension.

Step 1. Include Static TypoScript

  • Web > Templates > Root > TypoScript Edit > Include Static TypoScript
  • Search - Base Configuration (solr)
  • Search - Default Stylesheet (solr)

Step 2. Integrate TYPO3’s standard Search Markers

// TypoScript Setup
page.10 {
    stdWrap.dataWrap = <!--TYPO3SEARCH_begin-->|<!--TYPO3SEARCH_end-->
}

// Or you can configure at your Fluid Template too

Step 3. Configure at Site Handling

  • Sites Management > Sites > Edit your Site
  • Go to solr Tab
  • Configure all options available for TYPO3 solr extension
  • Go to Languages Tab
  • Configure “core_en” for solr Core
# sites/yoursite/config.yaml

base: 'http://yoursite.com/'
baseVariants: {  }
errorHandling: {  }
languages:
  -
    title: English
    enabled: true
    base: /
    typo3Language: default
    locale: en_US.UTF-8
    iso-639-1: en
    navigationTitle: ''
    hreflang: en-US
    direction: ''
    flag: global
    solr_host_read: solr_node_1
    solr_core_read: core_en
    languageId: '0'
  -
    title: German
    enabled: true
    base: /de/
    typo3Language: de
    locale: de_DE.UTF-8
    iso-639-1: de
    navigationTitle: ''
    hreflang: de-DE
    direction: ''
    flag: global
    solr_host_read: solr_node_2
    solr_core_read: core_de
    languageId: '1'
rootPageId: 3
routes: {  }
solr_enabled_read: true
solr_path_read: /solr/
solr_port_read: 8983
solr_scheme_read: http
solr_use_write_connection: false

First Time Index TYPO3 solr Extension

  • Go to Admin Tools > Search.
  • Click on Index Queue > Click on “Queue Selected Content for Indexing.”

Automize TYPO3 solr Indexing

  • Go to System > Scheduler
  • Add new task
  • Choose “Index Queue Worker” from Class
  • Configure other options of Scheduler

Step 1. Go to Web > Page > Choose Your Search Page

Step 2. Insert new content element > Plugins  > Search

Step 3. Choose “Search: Form, Result, Additional Components

Step 4. Configure Plugin Options

Step 5. Done! Go to your Frontend search result page.

Configure your custom TYPO3 Fluid template to TYPO3 solr Extension.

# TypoScript Setup
plugin.tx_solr {
    view {
        layoutRootPaths.10 = EXT:yourpath/Layouts/
        partialRootPaths.10 = EXT:yourpath/Partials/
        templateRootPaths.10 = EXT:yourpath/Templates/
    }
}

The extension provides such a luxurious feature in TYPO3 backend; You can explore available TYPO3 solr search backend modules as below.

Module 1. Connections

Module 2. Statistics

Module 3. Index Fields

Module 4. Stop Words

Module 5. Synonyms

Module 6. Index Queue

Module 7. Index Administration

Module 8. Search Index Inspector

1. Customize Autosuggest in TYPO3 solr Extension

# TypoScript Setup
plugin.tx_solr.search {
    suggest = 1
    suggest {
        numberOfSuggestions = 10
        suggestField = spell
        showTopResults = 1
        numberOfTopResults = 5
    }
}

2. Common Error “Search is currently not available.”

  • Should be configured your solr connection
  • Should empty <!– TYPO3SEARCH_begin –>” and “<!– TYPO3SEARCH_end –> in your template

3. Advanced Faces Content-Type

# TypoScript Setup
plugin.tx_solr.search.faceting.facets.[faceName].type = [typeName]

plugin.tx_solr.search {
    faceting = 1
    faceting {
        facets {
            contentType {
                label = Content Type
                field = type
            }
        }
    }
}

4. How to Index Custom Records in TYPO3 solr Extension?

# TypoScript Setup
plugin.tx_solr.index.queue {
    news = 1
    news {
        table = tx_news_domain_model_news
        fields {
            abstract = teaser
            author = author
            authorEmail_stringS = author_email
            title = title
            content = SOLR_CONTENT
            content {
                cObject = COA
                cObject {
                    10 = TEXT
                    10 {
                        field = bodytext
                        noTrimWrap = || |
                    }
                }
            }

            category_stringM = SOLR_RELATION
            category_stringM {
                localField = categories
                multiValue = 1
            }

            keywords = SOLR_MULTIVALUE
            keywords {
                field = keywords
            }

            tags_stringM = SOLR_RELATION
            tags_stringM {
                localField = tags
                multiValue = 1
            }

            url = TEXT
            url {
                typolink.parameter =  {$plugin.tx_news.settings.detailPid}
                typolink.additionalParams = &tx_news_pi1[controller]=News&
                    tx_news_pi1[action]=detail&tx_news_pi1[news]={field:uid}
                typolink.additionalParams.insertData = 1
                typolink.useCacheHash = 1
                typolink.returnLast = url
            }
        }

        attachments {
            fields = related_files
        }
    }
}

plugin.tx_solr.logging.indexing.queue.news = 1

And, make sure to disable search from the detail page.

5. How to Configure Search Result Per Page?

# TypoScript Setup
plugin.tx_solr {
    search {
        results {
            resultsPerPage = 6
            resultsPerPageSwitchOptions = 12, 18, 24
        }
    }
}

7. How can I configure sorting in TYPO3 solr?

# TypoScript Setup
plugin.tx_solr.search.sorting >
plugin.tx_solr.search {
        sorting = 1
        sorting {
                defaultOrder = asc

                options {
                        relevance {
                                field = relevance
                                label = Relevance
                        }

                        title {
                                field = sortTitle
                                label = Title
                        }
                }
        }
}

6. How can I exclude language in TYPO3 solr?

# TypoScript Setup
plugin.tx_solr.enabled = 0

[globalVar = GP:L = 0]
    plugin.tx_solr {
        enabled = 1
        solr.path = /solr/core_de/
    }
[globalVar = GP:L = 8|9]
    plugin.tx_solr {
        enabled = 1
        solr.path = /solr/core_en/
    }

8. How can I enable AJAX Search in TYPO3 solr?

Web > Templates > Root > Include static > Add “Search - Ajaxify the search results with jQuery (solr)”

The TYPO3 community developed a few good add-on TYPO3 solr extensions as follows.

1. TYPO3 solr Find

This TYPO3 extension aims to enable the query and display of arbitrary solr indexes. It provides the ability to configure many aspects of the query – e.g. query fields, facets, filtering through TypoScript – and set up the display through Fluid templates. Partials for standard display features as well as several View Helpers that help to create those templates are included in the extension.

2. solr file indexing

This extension gives you the capability to index individual documents using solr. Apache Tika, which is capable of detecting and extracting metadata from approx. 1200 different file types are used for document content analysis. The configuration for Tika can be implemented directly within the extension, solr server functionality is then used for parsing. Individual documents can be added to the search index for the default language or any localization; likewise, site roots can be selected for which the document is too indexed.

3. DDEV Apache solr for TYPO3 System

Get going quickly with TYPO3 CMS and Apache solr.

ddev start
http://solr-ddev-site.ddev.site/
http://solr-ddev-site.ddev.site/typo3/

By d way, you can also configure Apache solr for DDEV from their official guideline https://ddev.readthedocs.io/en/stable/users/extend/additional-services/

4. solr TYPO3 Plugin

The solr TYPO3 plugin provides the AccessFilterQParserPlugin, which is a org.apache.solr.search.QParserPlugin that allows restricting access to solr documents indexed by TYPO3 on a document level.

5. EXT:solr_file_indexer

This extension gives you the capability to index individual documents using solr.

Apache Tika, which is capable of detecting and extracting metadata from approx. 1200 different file types are used for document content analysis. The configuration for Tika can be implemented directly within the extension, solr server functionality is then used for parsing.

6. Apache Tika for TYPO3

Provides Tika services for TYPO3 to detect a document's language, extract metadata, and extract content from files. Can either use a stand-alone Tika executable or Tika integrated into a solr server with an activated extracting request handler.

7. JW tools

Jwtools2 contains a scheduler task for solr to index multiple Pages Trees and a job to execute SQL-Queries. Further, there are settings to enable some features in TYPO3 like showing the Page UID in Pagetree with a simple click in the extension manager.

Team DKD working hard to develop and maintain TYPO3 EXT:solr, You can help and contribute with the below ways.

Thanks for reading a bit long blog.

I hope you get little knowledge of how-to Apache TYPO3 solr search extension. Let’s quickly conclude as below.

  • Understand the concept, architecture, and installation of Apache solr.
  • Practice to install and configure awesome TYPO3 Apache solr Extension EXT:solr.
  • Study all available TYPO3 solr backend modules.
  • Test-drive and do customization of Frontend plugin with forms, search results, filter, etc. For detailed information, Checkout official documentation of EXT:solr, which is fantastic.
  • Try other available TYPO3 solr extensions from TER and TYPO3 solr Github.
  • As to give back to the community, Help by a contribution of sponsoring to EXT:solr project to great DKD guys.

Do you face any issues to install, configure, or use EXT:solr? I’ll be happy to help you, please write down our questions in the comment box.

Have a Happy TYPO3 solr Search!

Would you prefer a Ready-made template(s) or a Custom one for TYPO3?

  • 200+Developed custom TYPO3 extensions
  • 90+ Score on TYPO3 SEO, speed & accessibility
  • 70+ Published free & premium TYPO3 extensions
  • 13+Years of TYPO3 experience
  • 5+Bestseller TYPO3 extensions (T3AI, Slider Revolution etc.)
TYPO3 Extension Gig
Extension

Post a Comment

×
Captcha Code Can't read the image? Click here to refresh
  • user
    Fritz Peukert 2020-08-07 at 12:37 pm
    Thanks for the deep insghts Sanjay. Solr is simple enough to implement in TYPO3 without too much fuss.