Jeff Geerling's Blog: Re-save all nodes of a particular type in an update hook in Drupal 8

Planet Drupal - Mon, 2017/01/16 - 11:33pm

I recently needed to re-save all the nodes of a particular content type (after I had added some fields and default configuration) as part of a Drupal 8 site update and deployment. I could go in after deploying the new code and configuration, and manually re-save all content using the built-in bulk operation available on the /admin/content page, but that would not be ideal, because there would be a period of time where the content isn't updated on the live site—plus, manual processes are fragile and prone to failure, so I avoid them at all costs.

In my Drupal 8 module, called custom, I added the following update hook, inside custom.install:

Categories:

Joachim's blog: Changing the type of a node

Planet Drupal - Mon, 2017/01/16 - 11:22pm

There’s an old saying that no information architecture survives contact with the user. Or something like that. You’ll carefully design and build your content types and taxonomies, and then find that the users are actually not quite using what you’ve built in quite the way it was intended when you were building it.

And so there comes a point where you need to grit your teeth, change the structure of the site’s content, and convert existing content.

Back on Drupal 7, I wrote a plugin for Migrate which handled migrations within a single Drupal site, so for example from nodes to a custom entity type, or from one node type to another. (The patch works, though I never found the time to polish it sufficiently to be committed.)

On Drupal 8, without the time to learn the new version of Migrate, I recently had to cobble something together quickly.

Fortunately, this was just changing the type of some nodes, and where all the fields were identical on both source and destination node types. Anything more complex would definitely require Migrate.

First, I created the new node type, and cloned all its fields from the old type to the new type. Here I took the time to update some of the Field Tools module’s functionality to Drupal 8, as it pays off to have a single form to clone fields rather than have to add them to the new node type one by one.

Field Tools also copies display settings where form and view modes match (in other words, if the source bundle has a ‘teaser’ display mode configured, and the destination also has a ‘teaser’ display mode that’s enabled for custom settings, then all of the settings for the fields being cloned are copied over, with field groups too).

With all the new configuration in place, it was now time to get down to the content. This was plain and simple a hack, but one that worked fine for the case in question. Here’s how it went…

We basically want to change the bundle of a bunch of nodes. (Remember, the ‘bundle’ is the generic name for a node type. Node types are bundles, as taxonomy vocabularies are bundles.) The data for a single node is spread over lots of tables, and most of these have the bundle in them.

On Drupal 8 these tables are:

  • the entity base table
  • the entity data table
  • the entity revision data table
  • each field data table
  • each field data revision table

(It’s not entirely clear to me what the separation between base table and data table is for. It looks like it might be that base table is fields that don’t change for revisions, and data table is for fields that do. But then the language is on the base table, and that can be changed, and the created timestamp is on the data table, and while you can change that, I wouldn’t have thought that’s something that has past values kept. Answers on a postcard.)

So we’re basically going to hack the bundle column in a bunch of tables. We start by getting the names of these tables from the entity type storage:

$storage = \Drupal::service('entity_type.manager')->getStorage('node'); // Get the names of the base tables. $base_table_names = []; $base_table_names[] = $storage->getBaseTable(); $base_table_names[] = $storage->getDataTable(); // (Note that revision base tables don't have the bundle.)

For field tables, we need to ask the table mapping handler:

$table_mapping = \Drupal::service('entity_type.manager')->getStorage('node') ->getTableMapping(); // Get the names of the field tables for fields on the service node type. $field_table_names = []; foreach ($source_bundle_fields as $field) { $field_table = $table_mapping->getFieldTableName($field->getName()); $field_table_names[] = $field_table; $field_storage_definition = $field->getFieldStorageDefinition(); $field_revision_table = $table_mapping ->getDedicatedRevisionTableName($field_storage_definition); // Field revision tables DO have the bundle! $field_table_names[] = $field_revision_table; }

(Note the inconsistency in which tables have a bundle field and which don’t! For that matter, surely it’s redundant in all field tables? Does it improve the indexing perhaps?)

Then, get the IDs of the nodes to update. Fortunately, in this case there were only a few, and it wasn’t necessary to write a batched hook_update_N().

// Get the node IDs to update. $query = \Drupal::service('entity.query')->get('node'); // Your conditions here! // In our case, page nodes with a certain field populated. $query->condition('type', 'page'); $query->exists(‘field_in_question’); $nids = $query->execute();

And now, loop over the lists of tables names and hack away!

// Base tables have 'nid' and 'type' columns. foreach ($base_table_names as $table_name) { $query = \Drupal\Core\Database\Database::getConnection('default') ->update($table_name) ->fields(['type' => 'service']) ->condition('nid', $service_nids, 'IN') ->execute(); } // Field tables have 'entity_id' and 'bundle' columns. foreach ($field_table_names as $table_name) { $query = \Drupal\Core\Database\Database::getConnection('default') ->update($table_name) ->fields(['bundle' => 'service']) ->condition('entity_id', $service_nids, 'IN') ->execute(); }

Node-specific tables use ‘nid’ and ‘type’ for their names, because those are the base field names declared in the entity type class, whereas Field API tables use the generic ‘entity_id’ and ‘bundle’. The mapping between these two is declared in the entity type annotation’s entity_keys property.

This worked perfectly. The update system takes care of clearing caches, so entity caches will be fine. Other systems may need a nudge; for instance, Search API won’t notice the changed nodes and its indexes will literally need turning off and on again.

Though I do hope that the next time I have to do something like this, the amount of data justifies getting stuck into using Migrate!

Categories:

Dries Buytaert: Acquia retrospective 2016

Planet Drupal - Mon, 2017/01/16 - 7:30pm

As my loyal blog readers know, at the beginning of every year I publish a retrospective to look back and take stock of how far Acquia has come over the past 12 months. If you'd like to read my previous annual retrospectives, they can be found here: 2015, 2014, 2013, 2012, 2011, 2010, 2009. When read together, they provide a comprehensive overview of Acquia's trajectory from its inception in 2008 to where it is today, nine years later.

The process of pulling together this annual retrospective is very rewarding for me as it gives me a chance to reflect with some perspective; a rare opportunity among the hustle and bustle of the day-to-day. Trends and cycles only reveal themselves over time, and I continue to learn from this annual period of reflection.

Crossing the chasm

If I were to give Acquia a headline for 2016, it would be the year in which we crossed the proverbial "chasm" from startup to a true leader in our market. Acquia is now entering its ninth full year of operations (we began commercial operations in the fall of 2008). We've raised $186 million in venture capital, opened offices around the world, and now employ over 750 people. However, crossing the "chasm" is more than achieving a revenue target or other benchmarks of size.

The "chasm" describes the difficult transition conceived by Geoffrey Moore in his 1991 classic of technology strategy, Crossing the Chasm. This is the book that talks about making the transition from selling to the early adopters of a product (the technology enthusiasts and visionaries) to the early majority (the pragmatists). If the early majority accepts the technology solutions and products, they can make a company a de facto standard for its category.

I think future retrospectives will endorse my opinion that Acquia crossed the chasm in 2016. I believe that Acquia has crossed the "chasm" because the world has embraced open source and the cloud without any reservations. The FUD-era where proprietary software giants campaigned aggressively against open source and cloud computing by sowing fear, uncertainty and doubt is over. Ironically, those same critics are now scrambling to paint themselves as committed to open source and cloud architectures. Today, I believe that Acquia sets the standard for digital experiences built with open source and delivered in the cloud.

When Tom (my business partner and Acquia CEO) and I spoke together at Acquia's annual customer conference in November, we talked about the two founding pillars that have served Acquia well over its history: open source and cloud. In 2008, we made a commitment to build a company based on open source and the cloud, with its products and services offered through a subscription model rather than a perpetual license. At the time, our industry was skeptical of this forward-thinking combination. It was a bold move, but we have always believed that this combination offers significant advantages over proprietary software because of its faster rate of innovation, higher quality, freedom from vendor lock-in, greater security, and lower total cost of ownership.

Creating digital winners

Acquia has continued its evolution from a content management company to a company that offers a more complete digital experience platform. This transition inspired an internal project to update our vision and mission accordingly.

In 2016, we updated Acquia's vision to "make it possible for dreamers and doers to craft the digital world". To achieve this vision, we want to build "the universal platform for the world's greatest digital experiences".

We increasingly find ourselves at the center of our customer's technology and digital strategies, and they depend on us to provide the open platform to integrate, syndicate, govern and distribute all of their digital business.

The focus on any and every part of their digital business is important and sets us apart from our competitors. Nearly all of our competitors offer single-point solutions for marketers, customer service, online commerce or for portals. An open source model allows customers to integrate systems together through open APIs, which enables our technology to fit into any part of their existing environment. It gives them the freedom to pursue a best-of-breed strategy outside of the confines of a proprietary "marketing cloud".

Business momentum

We continued to grow rapidly in 2016, and it was another record year for revenue at Acquia. We focused on the growth of our recurring revenue, which includes new customers and the renewal and expansion of our work with existing customers. Ever since we started the company, our corporate emphasis on customer success has fueled both components. Successful customers mean renewals and references for new customers. Customer satisfaction remains extremely high at 96 percent, an achievement I'm confident we can maintain as we continue to grow.

In 2016, the top industry analysts published very positive reviews based on their dealings with our customers. I'm proud that Acquia made the biggest positive move of all vendors in this year's Gartner Magic Quadrant for Web Content Management. There are now three distinct leaders: Acquia, Adobe and Sitecore. Out of the leaders, Acquia is the only player that is open-source and has a cloud-first strategy.

Over the course of 2016 Acquia welcomed an impressive roster of new customers who included Nasdaq, Nestle, Vodafone, iHeartMedia, Advanced Auto Parts, Athenahealth, National Grid UK and more. Exiting 2016, Acquia can count 16 of the Fortune 100 among its customers.

Digital transformation is happening everywhere. Only a few years ago, the majority of our customers were in either government, media and entertainment or higher education. In the past two years, we've seen a lot of growth in other verticals and today, our customers span nearly every industry from pharmaceuticals to finance.

To support our growth, we opened a new sales office in Munich (Germany), and we expanded our global support facilities in Brisbane (Queensland, Australia), Portland (Oregon, USA) and Delhi (India). In total, we now have 14 offices around the world. Over the past year we have also seen our remote workforce expand; 33 percent of Acquia's employees are now remote. They can be found in 225 cities worldwide.

Acquia's offices around the world. The world got more flat for Acquia in 2016.

We've also seen an evolution in our partner ecosystem. In addition to working with traditional Drupal businesses, we started partnering with the world's most elite digital agencies and system integrators to deliver massive projects that span dozens of languages and countries. Our partners are taking Acquia and Drupal into some of the world's most impressive brands, new industries and into new parts of the world.

Growing pains and challenges

I enjoy writing these retrospectives because they allow me to chronicle Acquia's incredible journey. But I also write them for you, because you might be able to learn a thing or two from my experiences. To make these retrospectives useful for everyone, I try to document both milestones and difficulties. To grow an organization, you must learn how to overcome your challenges and growing pains.

Rapid growth does not come without cost. In 2016 we made several leadership changes that will help us continue to grow. We added new heads of revenue, European sales, security, IT, talent acquisition and engineering. I'm really proud of the team we built. We exited 2016 in the market for new heads of finance and marketing.

Part of the Acquia leadership team at The Lobster Pool restaurant in Rockport, MA.

We adjusted our business levers to adapt to changes in the financial markets, which in early 2016 shifted from valuing companies almost solely focused on growth to a combination of growth and free cash flow. This is easier said than done, and required a significant organizational mindshift. We changed our operating plan, took a closer look at expanding headcount, and postponed certain investments we had planned. All this was done in the name of "fiscal fitness" to make sure that we don't have to raise more money down the road. Our efforts to cut our burn rate are paying off, and we were able to beat our targets on margin (the difference between our revenue and operating expenses) while continuing to grow our top line.

We now manage 17,000+ AWS instances within Acquia Cloud. What we once were able to do efficiently for hundreds of clients is not necessarily the best way to do it for thousands. Going into 2016, we decided to improve the efficiency of our operations at this scale. While more work remains to be done, our efforts are already paying off. For example, we can now roll out new Acquia Cloud releases about 10 times faster than we could at the end of 2015.

Lastly, 2016 was the first full year of Drupal 8 availability (it was formally released in November 2015). As expected, it took time for developers and the Drupal community to become familiar with its vast array of changes and new capabilities. This wasn't a surprise; in my DrupalCon keynotes I shared that I expected Drupal 8 to really take off in Q4 of 2016. Through the MAP program we committed over $1M in funds and engineering hours to help module creators upgrade their modules to Drupal 8. All told, Acquia invested about $2.5 million in Drupal code contributions in 2016 alone (excluding our contributions in marketing, events, etc). This is the most we have ever invested in Drupal and something is I'm personally very proud of.

Product milestones

The components and products that make up the Acquia Platform.

Acquia remains an amazing place for engineers who want to build great products. We achieved some big milestones over the course of the year.

One of the largest milestones was the significant enhancements to our multi-site platform: Acquia Cloud Site Factory. Site Factory allows a team to manage and operate thousands of sites around the world from a single console, ensuring all fixes, upgrades and improvements are delivered responsibly and efficiently. Last year we added support for multiple codebases in Site Factory – which we call Stacks – allowing an organization to manage multiple Site Factories from the same administrative console and distribute the operation around the world over multiple data centers. It's unique in its ability and is being deployed globally by many multinational, multi-brand consumer goods companies. We manage thousands of sites for our biggest customers. Site Factory has elevated Acquia into the realm of very large and ambitious digital experience delivery.

Another exciting product release was the third version of Acquia Lift, our personalization and contextualization tool. With the third version of Acquia Lift, we've taken everything we've learned about personalization over the past several years to build a tool that is more flexible and easier to use. The new Lift also provides content syndication services that allow both content and user profile data to be reused across sites. When taken together with Site Factory, Lift permits true content governance and reuse.

We also released Lightning, Acquia's Drupal 8 distribution aimed at developers who want to accelerate their projects based on the set of tested and vetted modules and configurations we use ourselves in our customer work. Acquia's commitment to improving the developer experience also led to the release of both Acquia BLT and Acquia Pipelines (private beta). Acquia BLT is a development tool for building new Drupal projects using a standard approach, while Pipelines is a continuous delivery and continuous deployment service that can be used to develop, test and deploy websites on Acquia Cloud.

Acquia has also set a precedent of contributing significantly to Drupal. We helped with the release management of Drupal 8.1 and Drupal 8.2, and with the community's adoption of a new innovation model that allows for faster innovation. We also invested a lot in Drupal 8's "API-first initiative," whose goal is to improve Drupal's web services capabilities. As part of those efforts, we introduced Waterwheel, a group of SDKs which make it easier to build JavaScript and native mobile applications on top of Drupal 8's REST-backend. We have also been driving usability improvements in Drupal 8 by prototyping a new UX paradigm called "outside in" and by contributing to the media and layout initiatives. In 2017, I believe we should maintain our focus on release management, API-first and usability.

Our core product, Acquia Cloud, received a major reworking of its user interface. That new UI is a more modern, faster and responsive user interface that simplifies interaction for developers and administrators.

The new Acquia Cloud user interface released in 2016.

Our focus on security reached new levels in 2016. In January we secured certification that we complied with ISO 27001: the international security and compliance standard for enterprise cloud frameworks. In April we were awarded our FedRAMP ATO from the U.S. Department of Treasury after we were judged compliant with the U.S. federal standards for cloud security and risk management practices. Today we have the most secure, reliable and agile cloud platform available.

We ended the year with an exciting partnership with commerce platform Magento that will help us advance our vision of content and commerce. Existing commerce platforms have focused primarily on the transactions (cart systems, payment processing, warehouse/supply chain integration, tax compliance, customer credentials, etc.) and neglected the customer's actual shopping experience. We've demonstrated with numerous customers that a better brand experience can be delivered with Drupal and Acquia Lift alongside these existing commerce platforms.

The wind in our sales (pun intended)

Entering 2017, I believe that Acquia is positioned for long-term success. Here are a few reasons why:

  • The current market for content, commerce, and community-focused digital experiences is growing rapidly at just under 20 percent per year.
  • We hold a leadership position in our market, despite our relative market share being small. The analysts gave Acquia top marks for our strategic roadmap, vision and execution.
  • Digitization is top-of-mind for all organizations and impacts all elements of their business and value chain. Digital first businesses are seeking platforms that not only work for marketing, but also for service, compliance, portals, commerce and more.
  • Open source combined with the cloud continue to grow at a furious pace. The continuing rise of the developer's influence on technology selection also works in our favor.
  • Drupal 8 is the most significant advance in the evolution of the Drupal and Drupal's new innovation model allows the Drupal community to innovate faster than ever before.
  • Recent advances in machine learning, Internet of Things, augmented reality, speech technology, and conversational interfaces all coming to fruition will lead to new customer experiences and business models, reinforcing the need for API-first solutions and the levels of freedom that only open source and cloud computing offer.

As I explained at the beginning of this retrospective, trends and cycles reveal themselves over time. After reflecting on 2016, I believe that Acquia is in a unique position. As the world has embraced open source and cloud without reservation, our long-term commitment to this disruptive combination has put us at the right place at the right time. Our investments in expanding the breadth of our platform with products like Acquia Lift and Site Factory are also starting to pay off.

However, Acquia's success is not only determined by the technology we back. Our unique innovation model, which is impossible to cultivate with proprietary software, combined with our commitment to customer success has also contributed to our "crossing of the chasm."

Of course, none of these 2016 results and milestones would be possible without the hard work of the Acquia team, our customers, partners, the Drupal community, and our many friends. Thank you for your support in 2016 – I can't wait to see what the next year will bring!

Categories:

Evolving Web: Drupal 8 Migration: Migrating Basic Data (Part 1)

Planet Drupal - Mon, 2017/01/16 - 6:35pm

A tutorial on migrating basic data to a Drupal 8 site using the migrate module and related modules.

read more
Categories:

Web Wash: How to Manage Media Assets in Drupal 8

Planet Drupal - Mon, 2017/01/16 - 6:30pm
Everyone has their own definition of media management. In this tutorial, I'm going to focus on three parts: Storing assets, Embedding assets, Browsing assets. I want to give users the ability to create a media assets. Then have a button in the editor which they can use browse assets and then embed them. We'll utilize three modules to handle this: Media Entity, Entity Embed and Entity Browser.
Categories:

Drupal core announcements: Drupal 8 Migrate API is in beta

Planet Drupal - Mon, 2017/01/16 - 4:31pm

Drupal 8.0.0 replaced the earlier, in-place upgrade procedure for major version upgrades with a migration-based solution for core and contributed modules. Several modules serve this need in core: The Migrate module provides a general API for migrations, the Migrate Drupal module provides API support for Drupal-to-Drupal migrations, and the Migrate Drupal UI module offers a simple user interface to run migrations from older Drupal versions.

A lot of work has gone into making migrations more complete since the initial 8.0.0 release, including for multilingual sites with various configurations. Drupal-to-Drupal migrations are still not wholly complete (especially for Drupal 7 sources). However, lots of real-life use has validated the choices we made with the base Migrate API, and key architectural improvements have been completed already. An increasing number of contributed modules rely on it for their migrations.

Based on this stability and success, the Migrate subsystem maintainers and Drupal release managers have agreed the Migrate API (the Migrate module) now has beta stability! The change took effect in Drupal 8.2.x with 8.2.5 and will apply to 8.3.0 onwards as well.

What does this mean for sites and developers relying on the Migrate API?

Beta experimental modules are considered API- and feature-complete and beta modules are subject to the beta allowed changes policy. This means that module and migration developers can rely on the API remaining stable from now on! This also means that the focus with Migrate API is on fixing critical issues as well as bug fixes and contributed project blockers, if they are non-disruptive, or if the impact outweighs the disruption.

Note that Migrate Drupal and Migrate Drupal UI are still alpha stability, so API changes may still happen in these modules. Completing the Drupal-to-Drupal migration path and getting Migrate Drupal to beta stability is the next priority, so your help with missing and incomplete migrations is welcome!

If you want to get involved, the migration team is meeting every week at alternating times. The team has Google Hangouts at Wed 1pm UTC or Thu 1am UTC on an alternating weekly schedule. The #drupal-migrate IRC channel is also used as a backchannel.

Categories:

Sooper Drupal Themes: Introducing Glazed Builder: A Monumental Update By SooperThemes

Planet Drupal - Mon, 2017/01/16 - 3:53pm

2017 is going to be an incredible year for SooperThemes and for you! We kick off with the first official major version update for our drag and drop builder! Today we introduce you to Glazed Builder 1.1.0. We have rebranded Carbide Builder to Glazed Builder and developed some major new features, including entity revisions, full support for filtered contextual views and RGBA color sliders!

This is a monumental update. Glazed Builder is currently not only the greatest and fastest drag and drop builder for Drupal but also a very competitive product in the wider site building landscape. Going forward we'll not only develop more features and designs but we'll also start developing even more complete and complex turn-key Glazed demos. In addition this release marks the start of our Drupal 8 upgrade sprint. Yes, you heard that right, with the D8Media initiative aiming to add much needed Media improvements in Drupal 8.3 we feel like now is the right time. 

Try Glazed Builder Now

The Admin demo is free, no card required!

Redesigned for 2017

Glazed Builder has a fresh look that is designed for fast intuitive site-building. The speed of an interface isn't just about fast code, it's very much about fast design too. The new controls are all-white and while animation is used on the drop shadows the actual controls appear and respond immediately.

The colors and branding are matched to the Glazed brand, to emphasize the uniform experience you get when you combine our Glazed Theme with the builder. That said, Glazed Builder is still an independent module that can work with any Drupal theme! 

Views with Contextual Filters

Perfect if you're making a magazine website or more serious blog or commerce website! Now you can design taxonomy term or e-commerce pages with dynamic views displays. It is now fully supported to load views in drag and drop pages, override their filters and pagers in Glazed Builder, and let these views take default arguments based on url parts.

This feature is not just awesome for magazine websites but also for government websites, intranets, communities... Basically anyone who builds websites with large amounts of structured content will love this update.

Better Equal Heights Rows And Vertical Centering, RGBA Color

Having an alpha slider on every color setting in Glazed Builder allows more creative freedom for designers and makes it easier for site builders to implement designs that use translucent elements. We also improved equal heights rows by replacing the old jQuery trick with faster, reliable Flexbox CSS. In addition we enriched the column element with a Vertical Centering option. This let's you add vertical centering to columns independently of sibling columns. 

Saving Revisions

Some updates are invisible: Glazed Builder now automatically detects entity types that support revisions and will create new revisions when saving to fields on these entities. It doesn't matter if your field is on a node, bean block or custom entity. It doesn't even matter if you have have 10 Glazed Builder instances in one page, saving to multiple entities in the backend. It all just works and with revisions you'll easily undo those regrettable changes that seemed right in the spur of the moment.

​ Goole AMP

Google AMP is seeing more and more adoption and we integrated Glazed Builder with the Drupal AMP module. AMP is very restrictive so it won't allow fancy elements like circle charts and image comparison widgets but it should work fine with non-interactive markup generated by Glazed Builder. We welcome feedback from AMP experts on how to further extend AMP support!

Glazed Theme 2.6.0

While this releases focuses on the Glazed Builder update, Glazed Theme also received new features and consequently a major version bump to 2.6.0. Less spectaculary but still very useful: A Import/Export interface was added to the Glazed Theme Settings interface. All those settings that collectively make up your sites' unique design can now easily be copy-pasted between environments, or into a new subtheme .info file.

Drupal 8 Themes and Builder

        As a subscription Drupal theme shop, we really focus on building long-term relationships with our customers and that's why we carefully chose important moments that affect the products we make and maintain. Choosing the right moment to make the D8 move was hard, because Drupal 7 is just so good that it was hard for Drupal 8 to take the crown. 

        From my perspective, this moment is the release of Drupal 8.3 with contrib in great shape, improved media handling, and hopefully a media browser in core! The moment that it no longer makes sense for anyone to start a Drupal 7 website is approaching and we're going to make sure all our products are running smoothly on Drupal 8 by that time.

              source

              For more details about this update:

                    Try Glazed Builder Now

                    The Admin demo is free, no card required!

                    Categories:

                    ThinkShout: Using Google Docs and Migrate to Populate Your Drupal Site, Part 2

                    Planet Drupal - Mon, 2017/01/16 - 3:00pm

                    In Part 1, I talked about using Google Docs + Migrate to populate your site. Now we’re going to do that with the Migrate Google Sheets module. Below, I’ll provide the steps to get your own migration up and running, but if you prefer to experiment with a working example, check out a demo of the Migrate Google Sheets Example module (provided as a submodule within Migrate Google Sheets). All content on that site was built using this example Google Sheet.

                    Setup: Install the Module

                    If you’ve already got a Drupal 8 site up and running, you can install the module in any of the normal ways. I’m assuming here that you have access to the site using Drush, as it’s not possible to run migrations through anything but the command line at this time. At ThinkShout, we use composer to build our site distributions, and have a repo for building the demo site here.

                    Step 1: Creating Your Custom Migration Module

                    The easiest way to get started on your own set of migrations is to copy the migrate_google_sheets_example submodule and rename it something of your own. Let’s say we rename it “my_migration.” Follow these steps:

                    1. Rename your .install file to “my_migration.install”, and change the function migrate_google_sheets_example_uninstall to “my_migration_uninstall”.
                    2. Delete the helper submodule “migrate_google_sheets_example_setup” entirely – that is just necessary to build the content types required for the example module, but you shouldn’t need it for your migration module.
                    3. Rename your migrate_google_sheets_example.info.yml as “my_migration.info.yml” and open it up. At the very least, you’ll want to change the name of the migration to “name: my_migration” but you’ll also likely wish to remove the migrate_google_sheets:migrate_google_sheets_example_setup dependency. Mine ended up looking like this:
                    name: my_migration type: module description: My Migrations core: 8.x package: Migrate dependencies: - migrate_plus - migrate_tools - migrate_google_sheets - redirect

                    When completed, your module structure should look like this:

                    You are now ready to enable your My Migrations module. (Make sure you disable the migrate_google_sheets_example module first, if you’d previously enabled that!)

                    Step 2: Create Your Spreadsheet

                    Assuming you have the Game and Landing page content types, you could now run the migrations in your “My Migrations” module and it will pull the data from the Google Sheet.

                    But since you don’t have permissions to edit that sheet, you’re going to need to copy the existing sheet and create your own to do any customizations.

                    When this is done, you’ll get a url like this:

                    https://docs.google.com/spreadsheets/d/YourLongHashIDHere where YourLongHashIDHere is your feed key.

                    Now you’ll need to publish your new spreadsheet. This is an option under “File” -> “Publish to the web”

                    To verify that your migration module will be able to see the Google sheet, try opening an anonymous browser window and visiting the Feed version of the url, whose format is this:

                    https://spreadsheets.google.com/feeds/list/YourLongHashIDHere/SHEET/public/values?alt=json

                    If visiting that URL throws out a bunch of json, you’re ready to start migrating!

                    But of course, your current set of migration files still point to the old feed. In the my_migrations/config/install folder, you’ll need to find all instances of our feed string (1spS1BeUIzxR1KrGK2kKzAoiFZii6vBHyLx_SA0Sb89M) and replace them with your feed string.

                    Step 3: Decide Which Migrations to Keep

                    The Migrate Google Sheets Example module provides one Migration Group (games_example) and 6 Migrations. Depending on your site configuration, some of these might be useful, like the menu_links and the blocks migrations, and some of them will not be so useful (like the node_game migration, likely). This is a good time to trim or modify any migrations that aren’t going to be useful for your Drupal site. That being said, here are a few things that the sample migrations demonstrate:

                    • Block UUIDs: When you place a block using the Block Layout screen, the block’s UUID is saved in config. If you’re running a migration over and over, your block’s ID will iterate on its own, but the UUID can remain constant if you add it to the migration. In the demo site, this allows us to create a persistent CTA block in the header.

                    • Menu Links parents: You can specify that a menu link item has a parent from within the current migration. This lets us say /bohnanza and /hanabi are children of /games
                    • Page and Game redirects: These sheets demonstrate how to add the redirect from the url of content on an old site to the new home right in the content sheet. Try going to https://live-mgs-demo.pantheonsite.io/that-fireworks-game and see where you end up.
                    • Related content as strings or ids: On the Page sheet, we have a reference to the “Related games” for the given page. This is an entity reference which we could fill with a couple of things. We could refer to the ID of the related games, as they are stored in the Games sheet, or we could do what we’ve done here and use the migrate_plus plugin “entity_lookup” to search for the related game node by name. As long as there is a Game node called Bohnanza, we’ll always link to the right one. This is particularly useful with Term references, where the name of the item ideally remains constant.

                    • Game downloadable file: Games have associated images, which are files hosted externally to the spreadsheet. In order to relate my game content to its image, I need to download the image, get it into the file_managed database table (creating a file entity) and THEN relate that entity to the current node. This is done with the following lines in the “node_games” migration:
                    public_file_directory: plugin: default_value default_value: 'public://' public_file_uri: plugin: concat delimiter: '' source: - @public_file_directory - imagefilename field_image/target_id: - plugin: file_copy source: - image - @public_file_uri - plugin: entity_generate field_image/alt: imagealt field_image/title: imagetitle

                    You can keep as many or as few of the migration files as you’d like. You can also create new ones.

                    Step 4: Tell Drupal About Your Changes

                    Drupal 8 only sees the changes you’ve made to your migration yml files when you first install the module. That means that you need to uninstall and reinstall the module to make new changes show up. ThinkShout has a Robo script that does this, but the same thing can be done in Drush:

                    drush mr --all # Rolls back all migrations drush pmu my_migration -y # Disables my migration module drush en my_migration -y # Enable my migration module drush ms # Displays my current migration status

                    You can also string these together as one line:

                    drush mr --all && drush pmu my_migration -y && drush pmu my_migration -y && drush ms Step 5: Run your migrations

                    This part is simple. To run all migrations, it’s a quick drush command:

                    drush mi --all

                    If you’d like to find out about all the possible options for the migrate-import command, you can run

                    drush help mi

                    You can also see your list of migration groups at /admin/structure/migrate, and you can review your migrations by clicking “List Migrations.” The resulting page will give you the same output, more or less, that you get from running a drush ms.

                    These pages are helpful to know about, as they give you an easy place to find errors logged during the migration process. However, you can’t currently run a migration from the UI (although there is an issue for this).

                    Gotchas

                    But before we close, I do want to acknowledge some challenges we’ve seen in this approach.

                    Sad fact #1: HTML in a spreadsheet is ugly.

                    Google Spreadsheets don’t let you make your rows smaller than the number of line breaks in a cell. So if you have pretty html with a bunch of line breaks, your row might be too tall to fit on your screen. People have some clever workarounds for this, but so far we’ve not implemented any.

                    Sad fact #2: Sheet order matters (right now)

                    Maintaining the order of sheets isn’t top on everyone’s minds as they’re making changes to a spreadsheet, especially when duplicating tabs. Migrate Google Sheets asks for each sheet based on tab order. If I make a copy of the Page tab, the Game tab is now the fourth tab instead of the third tab.

                    As it stands now, the module will happily request columns that don’t exist on the third tab and then fail in puzzling ways.

                    There is currently only one issue in the issue queue for the Migrate Google Sheets module, and it’s to fix this.

                    Sad fact #3: Google sheets must be publicly viewable to work (again, right now)

                    As the module exists right now, there’s no authentication involved, so any migrated content must be publicly viewable. Google authorization is possible with Oauth2, but that is not currently implemented.

                    Conclusion

                    Thanks for following along! I hope you found this series helpful. And don’t forget to visit the Migrate Google Sheets issue queue if you find any bugs, have an idea for a feature, or need help!

                    Categories:

                    Cheppers blog: Development of a sponsored contrib module - Gather Content

                    Planet Drupal - Mon, 2017/01/16 - 10:52am

                    In 2016, our team worked on a sponsored contrib module, GatherContent. The goal of the project was to recreate the module for Drupal 7 using best practices, and to create a brand new module for Drupal 8.

                    Categories:

                    clemens-tolboom commented on issue VR-Tracker/Master-Control#2

                    On github - Sun, 2017/01/15 - 10:46am
                    Jan 15, 2017 clemens-tolboom commented on issue VR-Tracker/Master-Control#2

                    We were unable to display more then 2 tags while the data from all 4 tags were received.

                    ARREA-Systems: Custom view video tutorial: sales per project category

                    Planet Drupal - Sun, 2017/01/15 - 4:07am
                    Custom view video tutorial: sales per project category Sun, 01/15/2017 - 11:07

                    This video tutorial was made for our customer in order to demonstrate how to build a custom view to extract the data they need.

                    Here we are building a view to extract sales data per project where each project is classified into a category. We link invoice table with project table and add filter to be able to view data by year and category of project.

                    The tables and data sources used in this view are custom tables from our back office management solution built on Drupal 8. However, the principles of building a view are applicable to any other data source and this tutorial can be used to learn simple view building with tables relationships and filter.

                    Your browser doesn't support HTML5 video tag.

                    Categories:

                    Jeff Geerling's Blog: Can't upload more than 20 files using Media Image Entities in Drupal 8?

                    Planet Drupal - Sat, 2017/01/14 - 10:56pm

                    After migrating an older Drupal 6 site with 20,000 media items to Drupal 8, I found a strange problem with image uploads. On the Drupal site, using Image FUpload and Adobe Flash, I could upload up to 99 images in one go. On the new Drupal 8 site, I was only able to upload 20 images, even though I didn't see an error message or any other indication that the rest of the images I had selected through the Media Image upload form were not successfully added.

                    I could choose 21, 40, or 500 images, but only 20 were ever added to an album at any time.

                    There were no apparent warnings on the screen, so I just assumed there was some random bug in the Media Image Entity or Media module suite that limited uploads to 20 files at a time.

                    But due to an unrelated error, I glanced at the PHP logs one day, and noticed the following error message:

                    Categories:

                    Zhilevan Blog: Query on Drupal 8 with EntityQuery

                    Planet Drupal - Sat, 2017/01/14 - 1:32pm
                    Often when building a site in Drupal you'll find yourself wanting to display a list of nodes, or find entities created by a particular author, or locate some content based on a particular set of criteria. Rather than querying the database directly, Drupal provides a helper class, EntityQuery, to make things a bit easier.
                    Categories:

                    Frederic Marand: How to install a Drupal Composer based project when packages.drupal.org is down

                    Planet Drupal - Sat, 2017/01/14 - 11:55am
                    The problem: packages.drupal.org broken

                    I was starting my weekly work for the Drupal GraphQL module by the customary composer update --prefer-source -vvv command, ready to watch composer spit out some hundred lines sipping my coffee, but this time something turned out to be wrong:

                      [...snip...]
                    Downloading https://packages.drupal.org/8/drupal/provider-2016-4%24a30289dd8394e5271...

                      [Composer\Downloader\TransportException]
                      The "https://packages.drupal.org/8/drupal/provider-2016-4%24a30289dd8394e5271bd77777bb14b361c5938656f1cddad7fae1c00d5d6ba9c6.json" file could not be downloaded (HTTP/1.1 404 Not Found)
                      [...snip...]

                    OK, so packages.drupal.org is down for now. How can we work around this ?

                    read more

                    Categories:

                    Palantir: Palantir.net's Guide to Digital Governance

                    Planet Drupal - Fri, 2017/01/13 - 10:04pm
                    Palantir.net's Guide to Digital Governance Palantir.net's Guide to Digital Governance brandt Fri, 01/13/2017 - 15:04 Scott DiPerna Jan 16, 2017

                    This comprehensive guide is intended to help get you started when developing a governance plan for your institution’s digital communications.

                    In this guide we will cover...
                    • What digital governance is
                    • Why digital governance is important
                    • What topics you need to think about when building out a digital governance plan

                    We want to make your project a success.

                    Let's Chat.

                     

                    Download the guide here.

                    Stay connected with the latest news on web strategy, design, and development.

                    Sign up for our newsletter.
                    Categories:

                    Drupalize.Me: Experimental Module Changes Coming in Drupal 8.3

                    Planet Drupal - Fri, 2017/01/13 - 9:10pm

                    Drupal 8.3 is still a few months away, coming April 5, 2017, but there are already some changes we can look at, most notably in the experimental modules. In December, 2 new experimental modules were added to core, and BigPipe was officially changed from a beta module to stable. The 2 new modules you'll find in 8.3 are Workflows and Layout. Let's take a peek at what these are all about.

                    Categories:

                    Palantir: New Years Resolution: Get Organized

                    Planet Drupal - Fri, 2017/01/13 - 8:24pm
                    New Years Resolution: Get Organized brandt Fri, 01/13/2017 - 13:24 Allison Manley Jan 16, 2017

                    In this five-part series, every Monday in January we’ll explore a New Year’s resolution and how it can apply to your web project.

                    We want to make your project a success.

                    Let's Chat.

                    Unless you’re Marie Kondo of the KonMari Method and make your living organizing, you likely have a few messes somewhere. Photo albums? That pile of books? Year-end taxes and receipts? (‘Tis almost the season for that too.)

                    When it comes to your online space, here are a few tips we can offer on how to get organized for 2017:

                    Lastly, if you want a comprehensive look at how to get your internal governance in order, check out our detailed guide on digital governance (37 page PDF). The ownership, management, and sustainability plan for an organization’s various digital communications platforms seems simple at a distance, but the devil is in the details.

                    Next week’s resolution: stay fit and healthy.

                    Let Palantir help you get organized.

                    We'd love to help you keep your 2017 resolution.

                    Let's chat.
                    Categories:

                    DrupalCon News: Come to DrupalCon and _________ a Session About ___________

                    Planet Drupal - Fri, 2017/01/13 - 7:56pm

                    When I was a kid, my siblings and I used to play with Mad Libs. If you aren’t familiar with them, Mad Libs are basically one-page stories or vignettes, with various words missing. Where those words should be are blanks, and each blank is labeled “noun” or “verb” or whatever type of word should be filled in.

                    Categories:

                    Jeff Geerling's Blog: Testing redirections using Behat and the Behat Drupal Extension

                    Planet Drupal - Fri, 2017/01/13 - 7:33pm

                    One project I'm working on needed a Behat test added to test whether a particular redirection works properly. Basically, I wanted to test for the following:

                    1. An anonymous user accesses a file at a URL like http://www.example.com/pictures/test.jpg
                    2. The anonymous user is redirected to the path http://www.example.com/sites/default/files/pictures/test.jpg

                    Since the site uses Apache, I added the actual redirect to the site's .htaccess file in the docroot, using the following Rewrite configuration:

                    <IfModule mod_rewrite.c>
                      RewriteEngine on

                      # Rewrite requests for /profile_images to public files directory location.
                      RewriteRule ^ pictures/(.*)$ /sites/default/files/pictures/$1 [L,NC,R=301]
                    </IfModule>

                    Testing with curl --head, I could see that the proper headers were set—Location was set to the correct redirected URL, and the response gave a 301. So now I had to add the Behat test.

                    Categories:

                    Modules Unraveled: New FREE Series! Drupal 8: Composer and Configuration Management

                    Planet Drupal - Fri, 2017/01/13 - 5:34pm

                    I'm releasing a new series today!!!!!

                    Over the last year, I've given a talk at DrupalCon, DrupalCorn Camp, and Drupal Camp Colorado all about using Composer and Configuration Management in Drupal 8. Those sessions were around 45 minutes, which is much too short to go in depth, and explain everything thoroughly.

                    This series is the answer to that issue.

                    There is just over 1hr 15min of content in 26 videos that fall pretty well into seven parts. Here's the outline:

                    Part 1: Intro
                    • Intro
                    Part 2: Installing Drupal 8 Locally
                    • Creating a New Drupal 8 Project Using the Composer Template
                    • Setting Up MAMP to Serve Your Site Locally
                    • Using xip.io for Local Device Testing
                    • Creating a Drush Alias
                    • Installing Drupal with Console
                    • Configuring settings.php and settings.local.php
                    • Committing Your Project to Git
                    Part 3: Using Composer
                    • Installing and Uninstalling Modules with Composer
                    • Installing the Dev Version of Modules
                    • Updating and Downgrading Modules
                    • Skipping Specific Module Versions
                    • Specifying Acceptable Version Ranges
                    • Enabling Modules with Drush and Deciding What Version Pattern to Use
                    Part 4: Configuration Management
                    • Setting the Config Directory in settings.php
                    • Exporting Config Locally
                    • Using the Configuration Installer Install Profile
                    Part 5: Installing Drupal 8 on a Remote Server
                    • Installing the Site on a Production Server with Composer
                    Part 6: Overriding Settings in Code with settings.local.php
                    • Setting up settings.local.php
                    • Changing the Site Name and Disabling CSS Aggregation in settings.local.php
                    • How to Enable Theme Debugging on Development Sites
                    • Overriding Module Configuration (Like Google Analytics) in settings.local.php
                    Part 7: Putting it all Together
                    • Configuring a Local Site and Exporting it's Configuration with Git
                    • Pulling Changes to a Remote Site (And some gotchas)
                    • Using Drush Shell Aliases to Make Development Easier
                    • Verifying the Changes Made on Local and Reflected on Live

                    I'm pretty excited to finally have this series out, and hope you enjoy it! Oh... did I mention that it's ABSOLUTELY FREE?!?! Well, it is! So, check it out now, and let me know what you think.

                    Tags: Drupal 8ComposerConfiguration ManagementLocal DevelopmentDrushGitShell-Aliasesplanet-drupal
                    Categories: