drunken monkey: Search API in Drupal 8 is about to be stable!

Planet Drupal - Wed, 2017/04/05 - 1:09pm

TL; DR: After more than three years of development (and several blog posts), the Drupal 8 port of the Search API module is finally getting a stable release! The current Beta 5 release will be the last Beta, and there should be a release candidate and the final, stable release within the next two weeks. So, test it now, if you haven't already, to help eliminate any remaining (major) bugs!

The current status in detail

Last Saturday, April 1, I committed the fix for issue #2682369: Fix problems with overridden config entities, the last (and incredibly complicated – dedicated blog post upcoming) release blocker[1] for the stable release. The day after, I created the final Beta 5 release to give everyone still a few more days for testing.

Unless testing of this new Beta release discovers a serious enough problem, the first release candidate (RC) will be released on the coming weekend, April 8 or 9. Finally, if that doesn't uncover any major problems, the first stable release, Search API 8.x-1.0, will follow the weekend after that, on April 15 or 16. And then: rejoicing, parties and a little shield icon, to make everyone feel better.

To be clear, though, the Search API has been pretty usable for a long time (at least a year, I guess – possibly more, depending on your requirements), and we didn't have any major bugs or problems (unless you used config overrides) for some time now. The release blockers were mostly due to desired framework changes which would have been more difficult to do after a stable release, especially with other modules with stable releases already building on it. There were also several usability problems classified as release blockers, which was probably a mistake in hindsight. Having a clearer UI is nice, and helps avoid floods of bug reports due to confused users – but it shouldn't delay a stable release, and possibly dissuade site builders from using a basically working module. It also shouldn't prevent us from getting Security Team coverage, of course.

One plea regarding testing this last Beta: In case you are using config overrides for search indexes or servers in your site's setup or development workflow, it would be very helpful if you could verify that this last release fixes all associated problems you might have had – indexing/deleting using the wrong server (settings), overridden values being written back to the database, etc. This was a recent and pretty complex fix, so it would be great to feel more confident about that.

Also perhaps note-worthy: I've used the last few days of Dev Days Seville in March to finally update the module's documentation to Drupal 8. While it's still not quite finished, especially the documentation for developers, it should already provide a pretty good guide to the module's functionality and should help to answer most questions for first-time (or even advanced) users.

What's new

Compared to Drupal 7, the Search API in Drupal 8 is a massive step forwards – we didn't just spend those three years twiddling our thumbs, after all! Here's a run-down of the most important improvements we made:

Indexes can contain more than one type of item/entity

This might have been the single largest restriction in the Drupal 7 Search API: indexes could only be created for a single entity type (or custom item type). If you wanted a site search across entity types, you had to either use the Multi-Index Searches modules or, since August 2015, use the "Multiple types" item type, essentially a workaround-type backport of the D8 functionality.

In Drupal 8, however, creating an index for multiple item types (now called "datasources") is baked right into the framework, all parts of the Search API will support this natively. Furthermore, changing or re-configuring (think: changing the content types of nodes indexed) datasources is also supported right by the framework, and all such changes will be handled without any need to manually correct tracking data, or clear the index, etc.

Overhaul of the "Fields" tab and functionality

Also a continuing source of frustration, even if mostly just a usability annoyance, the "Fields" tab has seen a major revamp. Instead of seeing all available properties right on one page (killing the web server in some cases, for particularly large sites) and enabling the ones you want, we've transitioned to a UI inspired by Views: we only list the fields that are currently indexed and use a second page/dialog for discovery of available properties and adding the ones desired. This is also a lot more convenient than the old "Add related fields" box for indexing fields of related entities or data structures.

Furthermore, it's now possible to add the same property more than once, eliminating the need to use the "Aggregated fields" work-around of creating duplicates of a field. You can also freely change the human-readable name and the machine name of a field, and some processor-generated fields (like "Rendered HTML output") even allow per-field configuration (so you could have two "Rendered HTML output" fields, with each using different view modes, for different purposes). Also, like Views, changes made to the fields (but only to them) won't be effective immediately, but need to be explicitly saved afterwards. This avoids triggering a reindex numerous times when changing fields.

Improved language/translation support

Language support has also been vastly improved. While multilingual Drupal 7 sites needed the Search API Entity Translations contrib modules (at least when using the "entity translation" system, not "content translation") for getting proper translation support, which then didn't work natively with some other contrib modules, support for Drupal 8's new translation system is included right in the Search API itself. Automatically, all entities will be indexed once for every one of their translations, and each entity-based search item is identified not only by entity ID, but also by its translation language. Furthermore, additional internal improvements also make it easier for other modules (e.g., Search API Solr Multilingual) to add additional language-specific functionality.

Processors are more powerful than ever

This item is something that's probably only apparent as a developer, but processors have become a lot more powerful, being able to influence a lot of different areas of the search system. For non-developers, the most notable change is probably that the previous distinction between "processors" and "data alterations" has been removed – but not only have processors gained all the functionality previously held by data alterations, several other functions have been added, too.

For site builders, though, the main benefit here (in addition to more useful processors potentially becoming available) is a cleaner, less confusing user interface.

Objects everywhere!
Also not really anything that non-developers will notice, but internally the Search API has fully embraced Drupal 8's move to object-oriented programming (OOP). While its Drupal 7 version was already among the most OOP-using modules available (I'm pretty sure), this gained even more momentum in this new version. Not only did we, of course, make use of all the new OOP-based APIs in Core, like entities, plugins, services, etc. – we also converted a lot of the internal data structures (which, like Core, often used "magic" array structures in Drupal 7) to properly classed objects (e.g., indexed items, search results, fields) and moved most helper functions to methods on service classes.
Great test coverage
The last item, which no-one ever gets really excited about, but is actually also a huge step forward, is that we have improved test coverage throughout the module tremendously – aided a lot, of course, by the corresponding improvements in Drupal Core and on drupal.org. This (hopefully) means a lot less bugs that go undiscovered, a lot less regressions when introducing new features, simpler, more confident refactoring of old code and consequently less technical debt.
Other projects

Search API has always relied on additional contrib modules to provide powerful search solutions – with just the Search API module, you wouldn't get far. So, what is the current status of related projects in Drupal 8?

  • First off, the Database Search module (which provides the "Database" backend for servers) has been moved back into the Search API project itself, so this is already completely stable and very thoroughly tested. So, if you have a smaller site and don't require any advanced functionality (as offered by Solr or Elasticsearch), by just downloading the Search API project you can already set up a pretty good database-based search.
  • The Facets module (replacement for the Drupal 7 Facet API module) is still in Alpha as there are still some API changes and breaks planned. However, its basic functionality is already there and quite extensive, and it's already used on quite a few production sites. So, if you want to add facetting to your search, you should at least give it a try.
  • The Search API Solr module is in a similar position: It's already used on a lot of production sites, but there's still some rough edges (especially regarding support of optional features) and the module is currently considered to be in Beta state. However, the test coverage is quite good, so there shouldn't be any larger bugs lurking under the surface and, if you want to use Solr for your site, you shouldn't hesitate to give it a try. (Just beware that, like all modules with no stable releases, it's currently not covered by the Drupal Security Team, so security issues are a larger risk than usual.)
  • If you want to use Elasticsearch, things look a bit worse. There are (confusingly) several modules in Drupal 7, but only Elasticsearch Connector has a Drupal 8 version yet. And since even the Drupal 7 version is not yet beyond Alpha (and the Drupal 8 version has nothing more than a dev snapshot release), it may take some time until it becomes stable. I haven't evaluated the code or functionality, though, so it might actually already work quite well – if you have information, please leave a comment below!
  • The Pages module has been ported to Drupal 8 (and is currently maintained) by swentel. It, too, is already used on some production sites and has most of its functionality working fine (though some adaptions might be needed for the lasted Search API changes). It's currently in Alpha state.
  • Preliminary ports also exist for both the Autocomplete and the Location module, with at least some functionality already working. The former of those will probably be my next focus, since it was one of the most popular extension modules in Drupal 7 (apart from the mostly-ported ones mentioned above). And the latter is set to be the focus of an upcoming Google Summer of Code project by dbjpanda, so we'll probably have a full-fledged Drupal 8 version of this by the end of August.
  • The Saved Searches module has not been ported yet. I plan to start porting it once the Autocomplete port has been finished and I get the time.

Once again, I have to state that this release wouldn't have been possible, and the development would have taken even longer, if it weren't for a lot of helping hands who did their part in moving this module port along. Chief among them are Acquia, who sponsored two months of my time a year ago to focus on this port, and Joris Vercammen (borisson_) who was an invaluable help just by discussing ideas and reviewing patches (and being probably the second-best informed regarding the Drupal 8 Search API module). Thanks to both of them, but also to everyone else who contributed – much too many to list them all!

[1] To be accurate, there are still two open but postponed ones, but those are mere clean-up issues that will be committed right before RC creation.

Image credit: bayasaa


Agiledrop.com Blog: AGILEDROP: Drupal Blogs from March

Planet Drupal - Wed, 2017/04/05 - 12:27pm
Our blog activities were very frequent in March. We have written many blog posts for you and now it's time to look at them, in case you have missed any. We started last month with Drupal Logos as Fruits and Vegetables. It was the second part of our »Druplicon marathon«, where we have analysed how Druplicons can be designed as fruits and vegetables. We have immediately received the missing ones in the field of Humans and Superhumans and a Buckeye tree fruit, which was mixed with a hedgehog. Without a hedgehog, our Drupal Logos in the shapes of Animals, were impoverished. Nevertheless, we… READ MORE

agoradesign: Why running \Drupal::entityDefinitionUpdateManager()->applyUpdates() in update hooks is bad

Planet Drupal - Wed, 2017/04/05 - 11:45am
In this blog post I explain why running \Drupal::entityDefinitionUpdateManager()->applyUpdates() in update hooks is very bad. I've already seen this a few times now, and today it was one of times it caused a problem during updating a site.

KatteKrab: Making Views in Drupal 7 and Drupal 8

Planet Drupal - Wed, 2017/04/05 - 3:45am
Wednesday, April 5, 2017 - 11:45

This talk was written for DrupalGov Canberra 2017.

Download a PDF of the slides.

Or view below on slideshare.


Making views - DrupalGov Canberra 2017 from Donna Benjamin AttachmentSize Making Views in Drupal 7 and Drupal 84.46 MB

OSTraining: The Bigger Picture Surrounding the Drupal Controversy

Planet Drupal - Tue, 2017/04/04 - 11:18pm

This has not been a fun couple of weeks in the Drupal community.

If you've missed the events, you can catch up with this list of links. There's dozens of articles in that list, plus hundreds more articles, tweets and Reddit posts.

You don't need any more opinion on what happened.

Instead, I'd like to try and put this controversy into the broader flow of events.


Chapter Three: Twig Concepts in Drupal 8 Themes - Part I

Planet Drupal - Tue, 2017/04/04 - 8:06pm

Twig is a powerful templating engine with many useful features. In the context of Drupal, especially when coming from a Drupal 7 way of doing things, it can be easy to overlook many of these features in the interest of just getting the job done quickly.  However, taking the time to learn Twig beyond just printing variables is worthwhile, because it can help you solve common problems.  

In this series, we will explore some of the tools Twig itself provides, within the context of a Drupal 8 theme to improve the quality of the code we write. Each post will identify a common problem, and provide details about Twig concepts that can be used to help solve them.


David Lohmeyer's Blog: Quick tool for generating Drupal hash salt strings

Planet Drupal - Tue, 2017/04/04 - 3:29pm

Every time I install Drupal I find myself fumbling for a nice hash salt string. Instead of finding a random solution, I decided to put together a quick page that generates a random hash salt string and get a shiny domain for it.

Check out DrupalHashSalt.com (easy to remember!) next time you need a random hash salt. I also posted the code used to generate the string.

If you have ideas for this site feel free to contact me.


Acquia Developer Center Blog: Mother of All Demos II: The Future of Acquia Cloud

Planet Drupal - Tue, 2017/04/04 - 3:26pm

One of the highlights of Acquia Engage is when Chief Product Officer Chris Stone and his team of engineers demo Acquia products and give attendees a preview of what is on the horizon. Dubbed “The Mother of All Demos,” this presentation showcases the best of Acquia’s products and offers insight into the future of the company, not just Acquia’s product offerings.

Tags: acquia drupal planet

Flocon de toile | Freelance Drupal: Token and chained entities on Drupal 8

Planet Drupal - Tue, 2017/04/04 - 12:00pm

The token module is one of these essential modules on any Drupal 8 project. It allows you to use tokens in certain input fields, whether configuration or content, to target the value of one entity field. Many modules use it to allow users or site builder to provide dynamic value without the need for coding.

Let's see how to access the content's values from these tokens, but also to the values ​​indirectly associated with these contents, from Entity reference fields.


Roman Agabekov: Making your robots.txt unaffected by Drupal core upgrade

Planet Drupal - Tue, 2017/04/04 - 11:28am
Making your robots.txt unaffected by Drupal core upgrade Submitted by admin on Tue, 04/04/2017 - 09:28

Today’s post is a quick tip that will help you keep your robots.txt after you update the site’s Drupal core with drush.

Most likely, you know the problem, too: each time you update the Drupal core (with drush, first of all), you lose all changes you may have introduced to robots.txt manually. A good idea is to always have a backup of this file handy, but every now and then you only have an outdated version of robots.txt and you learn that only when you check the updated site with webmaster tools.


Wunderkraut Sweden Blog: Cache in Drupal 8: How it worked, and what we wanted to be solved in D8

Planet Drupal - Tue, 2017/04/04 - 11:23am
If you just want your content to be cached before Drupal 8, there were almost no problems, just turn on caching for anonymous users, and you are all set. Muhahhaha! Who am I kidding... If you just want your content to be cached before Drupal 8, there were almost no problems, just turn on caching for anonymous users, and you are all set. Muhahhaha! Who am I kidding... If you want to interact with users with different content depending on the user, role etc. You got problems, if you want to invalidate the cache, you got problems. If you want to show a View of nodes, and turn off caching of that view, but had caching for anonymous users, you got problems. And so on. The most caching issues were solved by clearing all the cache, which could bring down the site if your unlucky. So the real problem we wanted solve were not caching per see, it was cache invalidation. And like Phil Karlton supposedly said in the early nineties “There are only two hard things in Computer Science: cache… Read More

Code Positive: Specification By Example For Drupal Agencies

Planet Drupal - Tue, 2017/04/04 - 9:35am

Bridging the communications gap between clients and Drupal developers with Specification By Example.


Code Positive: Specification By Example For Drupal Angencies

Planet Drupal - Tue, 2017/04/04 - 9:35am

Bridging the communications gap between clients and Drupal developers with Specification By Example.


Droptica: Droptica: Your own Drush command in Drupal 8

Planet Drupal - Tue, 2017/04/04 - 9:19am
In Drupal 8 we can easily add our own command to Drush. The whole code for the new command will be put in a custom module. We have to create only 2 files. The structure looks as follows. File d_drush_command_pack.info.yml A standard file of every module in which there is information about our plugin.

CU Boulder - Webcentral: Writing Node Migrations From Drupal 7 to Drupal 8

Planet Drupal - Tue, 2017/04/04 - 8:00am
Are Content Migrations for Drupal 8 As Easy As Advertised? Developer Blog

Acquia Developer Center Blog: Password Policies and Drupal

Planet Drupal - Mon, 2017/04/03 - 5:57pm

People tend to choose bad passwords if they are allowed to.

By default Drupal provides some guidance about how to "make your password stronger," but there's no enforcement of any particular password policy out of the box. As usual, there's a module for that. More than one in fact.

Tags: acquia drupal planet

Code Positive: Getting set up with Behat 3 on Windows

Planet Drupal - Mon, 2017/04/03 - 5:22pm

A step by step guide to installing Behat 3 for Windows.


Code Positive: Benefits of Behat

Planet Drupal - Mon, 2017/04/03 - 4:10pm

Behat stories are human-readable descriptions of how a website should behave, which can be used for automated testing.


Third & Grove: Third & Grove goes to DrupalCon Baltimore

Planet Drupal - Mon, 2017/04/03 - 11:59am
Third & Grove goes to DrupalCon Baltimore antonella Mon, 04/03/2017 - 05:59

SystemSeed: Apple Pay on Concern.net

Planet Drupal - Mon, 2017/04/03 - 9:35am

We are very happy to announce that after several months of combined work with Concern Worldwide and Apple we now support Apple Pay via desktop, iPhone, iPad and Apple Watch @ Concern.net!

This was a fascinating endeavour for our team with many requirements spanning both hardware and software.

We now have tools in place for Concern.net donation forms to enable existing donors to make a donation, or increase their regular donations by following a marketing link and press a single button on the form before using their fingerprint to authenticate payment.

There is a famous saying that “The best GUI is no GUI” This ideal is now a tangible possibility.

Stay tuned for further developments as to how you can donate without fuss to one of the world’s leading charities.

Our many thanks to all the teams at SystemSeed, Concern and Apple that helped facilitate this work.

Please consider making the world a brighter place for those in need by going to Concern.net and donating via Apple Pay or any other method today.