Revered management thinker Peter Drucker once wrote, “If you can’t replicate something because you don’t understand it, then it really hasn’t been invented; it’s only been done.” In many ways content modeling in Drupal has been done without being invented. For this reason, we’re developing a discipline for content modeling at Acquia. It’s drastically reducing both costs and defect rates for us.Tags: acquia drupal planet
As part of the Decoupled Hard Problems series, in this fourth article, I'll discuss some of the challenges surrounding routing, custom paths and URL aliases in decoupled projects.Decoupled Routing
It's a Wednesday afternoon, and I'm using the time that Lullabot gives me for professional development to contribute to Contenta CMS. Someone asks me a question about routing for a React application with a decoupled Drupal back-end, so I decide to share it with the rest of the Contenta Slack community and a lengthy conversation ensues. I realize the many tendrils that begin when we separate our routes and paths from a more traditional Drupal setup, especially if we need to think about routing across multiple different consumers.
It's tempting to think about decoupled Drupal as a back-end plus a JS front-end application. In other words, a website. That is a common use case, probably the most common. Indeed, if we can restrict our decoupled architecture to a single consumer, we can move as many features as we want to the server side. Fantastic, now the editors who use the CMS have many routing tools at their disposal. They can, for instance, configure the URL alias for a given node. URL aliases allow content editors to specify the route of a web page that displays a piece of content. As Drupal developers, we tend to make no distinction between such pieces of content and the web page that Drupal automatically generates for it. That's because Drupal hides the complexity involved in making reasonable assumptions:
- It assumes that we need a web page for each node. Each of those has a route node/<nid> and they can have a custom route (aka URL alias).
- It means that it is okay to add presentation information in the content model. This makes it easy to tell the Twig template how to display the content (like field_position = 'top-left') in order to render it as the editor intended.
Unfortunately, when we are building a decoupled back-end, we cannot assume that our pieces of content will be displayed on a web page, even if our initial project is a website. That is because when we eventually need a second consumer, we will need to make amends all over the project to undo those assumptions before adding the new consumer.
Understand the hidden costs of decoupling in full. If those costs are acceptable—because we will take advantage of other aspects of decoupling—then a rigorous separation of concerns that assigns all the presentation logic to the front-end will pay off. It takes more time to implement, but it will be worth it when the time comes to add new consumers. While it may save time to use the server side to deal with routing on the assumption that our consumer will be a single website, as soon as a new consumer gets added those savings turn into losses. And, after all, if there is only a website, we should strongly consider a monolithic Drupal site.undefined
After working with Drupal or other modern CMSes, it's easy to assume that content editors can just input what they need for SEO purposes and all the front-ends will follow. But let's take a step back to think about routes:
- Routes are critical only for website clients. Native applications can also benefit from them, but they can function with just the resource IDs on the API.
- Routes are important for deep linking in web and native applications. When we use a web search engine in our phone and click a link, we expect the native app to open on that particular content if we have it installed. That is done by mapping the web URL to the app link.
- Links are a great way to share content. We want users to share links, and then let the appropriate app on the recipient's mobile device open if they have it installed.
It seems clear that even non-browser-centric applications care about the routes of our consumers. Luckily, Drupal considers the URL alias to be part of the content, so it's available to the consumers. But our consumers' routing needs may vary significantly.Routing From a Web Consumer
Let's imagine that a request to http://cms.contentacms.io/recipes/4-hour-lamb-stew hits our React application. The routing component will know that it needs to use the recipes resource and find the node that has a URL alias of /4-hour-lamb-stew. Contenta can handle this request with JSON API and Fieldable Path—both part of the distribution. With the response to that query, the React app builds all the components and displays the results to the user.
It is important to note the two implicit assumptions in this scenario. The first is that the inbound URL can be tokenized to extract the resource to query. In our case, the URL tells us that we want to query the /api/recipes resource to find a single item that has a particular URL alias. We know that because the URL in the React side contains /recipes/... What happens if the SEO team decides that the content should be under https://cms.contentacms.io/4-hour-lamb-stew? How will React know that it needs to query the /api/recipes resource and not /api/articles?
The second assumption is that there is a web page that represents a node. When we have a decoupled architecture, we cannot guarantee a one-to-one mapping between nodes and pages. Though it's common to have the content model aligned with the routes, let's explore an example where that's not the case. Suppose we have a seasonal page in our food magazine for the summer season (accessible under /summer). It consists of two recipes, and an article, and a manually selected hero image. We can build that easily in our React application by querying and rendering the content. However, everything—except for the data in the nodes and images—lives in the react application. Where does the editor go to change the route for that page?
On top of that, SEO will want it so that when a URL alias changes (either editorially or in the front-end code) a redirect occurs, so people using the old URL can still access the content. Note that a change in the node title could trigger a change in the URL alias via Pathauto. That is a problem even in the "easy" situation. If the alias changes to https://cms.contentacms.io/recipes/four-hour-stewed-lamb, we need our React application to still respond to the old https://cms.contentacms.io/recipes/4-hour-lamb-stew. The old link may have been shared in social networks, linked to from other sites, etc. The problem is that there is no recipe with an alias of /recipes/4-hour-lamb-stew anymore, so the Fieldable Path solution will not cover all cases.Possible Solutions
In monolithic Drupal, we'd solve the aforementioned SEO issue by using the Redirect module, which keeps track of old path aliases and can respond to them with a redirect to the new one. In decoupled Drupal, we can use that same module along with the new Decoupled Router module (created as part of the research for this article).
Pages—or visualizations—that comprise a disconnected selection of entities—our /summer page example—are hard to manage from the back-end. A possible solution could be to use JSON API to query the entities generated by Page Manager. Another possible solution would be to create a content type, with its corresponding resource, specific for that presentation in that particular consumer. Depending on how specific that content type is for the consumer, that will take us to the Back-end For Front-end pattern, which incurs other considerations and maintenance costs.
For the case where multiple consumers claim the same route but have that route resolve to different nodes, we can try the Contextual Aliases module.The Decoupled Router
Decoupled Router is an endpoint that receives a front-end path and tries to resolve it to an entity. To do so it follows as many redirects and URL aliases as necessary. In the example of /recipes/four-hour-stewed-lamb it would follow the redirect down to /recipes/4-hour-lamb-stew and resolve that URL alias to node:1234. The endpoint provides some interesting information about the route and the underlying entity.undefined
In a previous post, we discussed how multiple requests degrade performance significantly. With that in mind, making an extra request to resolve the redirects and aliases seems less attractive. We can solve this problem using the Subrequests module. Like we discussed in detail, we can use response tokens to combine several requests in one.
Imagine that we want to resolve /bread and display the title and image. However, we don’t know if /bread will resolve into an article or a recipe. We could use Subrequests to resolve the path and the JSON API entity in a single request.undefined
In the request above, we provide the path we want to resolve. Then we get the following response.undefined
To summarize, we can use Decoupled Router in combination with Subrequests to resolve multiple levels of redirects and URL aliases and get the JSON API data all in a single request. This solution is generic enough that it serves in almost all cases.Conclusion
Routing in decoupled applications becomes challenging because of three factors:
- Instead of one route, we have to think about (at least) two, one for the front-end and one for the back-end. We can mitigate this by keeping them both in sync.
- Multiple consumers may decide different routing patterns. This can be mitigated by reaching an agreement among consumers. Another alternative is to use Contextual Aliases along with Consumers. When we want back-end changes that only affect a particular consumer, we can use the Consumers module to make that dependency explicit. See the Consumer Image Styles module—explained in a previous article—for an example of how to do this.
- Some visualizations in some of the consumers don’t have a one-to-one correspondence with an entity in the data model. This is solved by introducing dedicated content types for those visualizations. That implies that we have access to both back-end and front-end. A custom resource based on Page Manager could work as well.
In general, whenever we need editorial control we'll have to turn to the back-end CMS. Unfortunately, the back-end affects all consumers, not just one. That may or may not be acceptable, depending on each project. We will need to make sure to consider this when thinking through paths and aliases on our next decoupled Drupal project.
Lucky for us, every project has constraints we can leverage. That is true even when working on the most challenging back-end of all—a public API that powers an unknown number of 3rd-party consumers. For the problem of routing, we can leverage these constraints to use the mitigations listed above.
Hopefully, this article will give you some solutions for your Decoupled Drupal Hard Problems.
Note: This article was originally published on November 29, 2017. Following DrupalCon Nashville, we are republishing (with updates) some of our key articles on decoupled or "headless" Drupal as the community as a whole continues to explore this approach further. Comments from the original will appear unmodified.
The new GDPR laws are here, hurrah!
Having a number of developers handling databases from a number of client sites could easily be a nightmare, but we at ComputerMinds spent quite some time thinking about how to get and keep everybody safe and squeaky clean on the personal data front.
Here's a quick run-down of the key things to be aware of - and a pretty poster to help you keep it all in mind :)
Remove personal data from your system
- Review all databases on your computer, making sure to consider also those .sql dump files still sat in your downloads directory or your Recycle bin/trash.
- If there are databases that you need to keep on your system, then you must sanitize them by encrypting, anonymizing or removing personal data.
- Review all testing / UAT environments and ensure they're running off sanitized databases where possible.
Stay clean by using sanitized databases
Set up some _drush_sql_sync_sanitize() hooks to deal with personal data stored on your site. Then either have your Jenkins server use it to provide sanitized dumps, or ensure that your developers use it to sanitize databases immediately after importing.
When setting up your hook, make sure to consider things like:
- User table - clear out email addresses, usernames etc.
- Custom fields on users - names, telephone numbers etc. that you've added.
- Webform / contact form submissions - make sure that your Webform / contact form data gets cleared out. Webform 7.12 and above has these hooks included, but it's good to double-check.
- Commerce order table - you'll need to remove personal data from the commerce orders.
- Commerce profile tables - make sure that the personal data in the profiles gets anonymized or removed.
- Commerce payment gateway callback tables - these will have detailed payment transaction data, and absolutely must be cleared out.
- URL aliases & redirects - by default Drupal sets up aliases for users' usernames, so you'll need to review those tables.
- Comments - these usually have name, email and website fields that will need clearing out. But their body content may also have personal data in too, so you might be better off just binning the lot.
- Watchdog / logging tables - these take up lots of space, so you probably don't want to export them off the live site anyway, but think seriously about the personal data inside if you do decide you want to import them elsewhere. Truncate recommended.
- Cache tables - these can be huge, so you probably don't want to export them off the live site anyway, but think seriously about the personal data inside if you do decide you want to import them elsewhere. Truncate recommended.
This is certainly not a complete list, but we can't tell you what custom fun you've implemented on your site - so its' down to you to go check your tables!
- Ensure future development environments and UAT/test environments are built using sanitized databases.
- If you receive user data via email, immediately delete the email and attachments and reprimand the sender!
- Talk to your clients about changes that need to be made to their sites.
PDF download link below!
Firefox 60 was released a few weeks ago and now comes with support for the upcoming Web Authentication (WebAuthn) standard.
Other major web browsers weren't far behind. Yesterday, the release of Google Chrome 67 also included support for the Web Authentication standard.
I'm excited about it because it can make the web both easier and safer to use.
Supporting for the Web Authentication standard will make the web easier, because it is a big step towards eliminating passwords on the web. Instead of having to manage passwords, we'll be able to use web-based fingerprints, facial authentication, voice recognition, a smartphone, or hardware security keys like the YubiKey.
It will also make the web safer, because U2F will help reduce or even prevent phishing, man-in-the-middle attacks, and credential theft. If you are interested in learning more about the security benefits of the Web Authentication standard, I recommend reading Adam Langley's excellent analysis.
When I have a bit more time for side projects, I'd like to buy a YubiKey 4C to see how it fits in my daily workflow, in addition to what it would look like to add Web Authentication support to Drupal and https://dri.es.
TEN7 Blog's Drupal Posts: Episode 029: Wilbur Ince, Drupal Frontend Developer and Human Rights Activist
Estimation is not a new concept nor is it a bad word.
Estimation is not a new topic for anyone in the Drupal or open source community. We do it every day at our jobs. We even discuss estimation techniques at our conferences. We provide our clients with estimates when building and contributing back to open source project, yet we don't include estimations within our open source community and issue queues.
We all provide estimates to our clients - are we afraid to do it when it comes to Drupal and Open Source?
Before we take on this tough question, 'Are we afraid to estimate our work in Drupal and open source?', let's start off with a straightforward question: 'Why do we provide estimates to our clients?' The answer is just that our clients want to know how much something is going to cost and we want to know how much work is required to complete a project.
To give this discussion more context, let's begin with a very general definition of estimation
Here’s my hypothesis:
The Science of guessing - Drupal estimation techniques from project managers
While researching estimation within the Drupal community, I found a bunch of great presentations about project management and estimation. To me, "The Science of guessing - Drupal estimation techniques from project managers" by Shannon Vettes (svettes), Jakob Persson (solipsist), and Mattias Axelsson (acke), was the most comprehensive and inspiring presentation. Feel free to watch this presentation. I am going to pull a few slides from this presentation to help move through this exploration.
Every presentation I watched focused on estimation concerning managing expectations for...Read More
Drupal module - CiviCRM Contact Distance Search
MillerTech released this Drupal module back in 2015 but have recently updated with new features (map and use your location) and to make it more configurable.
This module offers a fully configurable/extendable Drupal view that provides the functionality to search from a postcode and a distance.
Use case scenario – Find schools from my postcode within a 5 mile radius.
With the example above you would have schools as contacts in your CiviCRM database with a primary address and both the latitude and longitude fields should be populated.
The Drupal view that’s shipped with this module can be configured to filter on a particular contact subtype i.e. schools.
Search results will provide you with schools within a 5 mile radius of the entered postcode along with distance.
Distance is calculated by road (or as the road winds or as the crow walks etc.) and NOT as the crow flies.
New features includes an option to display a map –
And also an option for your device to use your location which will populate the postcode field (works best with mobile devices for accuracy) –
CiviCRM extension page - https://civicrm.org/extensions/civicrm-contact-distance-search
Full installation steps available on the Drupal module page - https://www.drupal.org/project/civicrm_contact_distance_search
The Promote Drupal Initiative is your opportunity to make Drupal - and your business - known and loved by new decision makers. Donate to the Promote Drupal Fund today. Help us help you grow your business.
Together, let's show the world just how amazing Drupal is for organizations.
We are 70% to the $100,000 goal. Help us reach the goal.
Donate to the Promote Drupal Fund today. Invest $1,000 or more and be highlighted in:
Dries’ blog post once we reach 75% of goal
Dries’ presentation at Frontend United
This is the final installment of Drupal Taxonomy that we feel is important for one unfamiliar with Drupal to know! At this point, hopefully, you understand some of the key language that is regularly used in the Drupal Community. For reference, our first two blogs, Part 1 and Part 2, should provide you any background you may not already have. Mobomo is the team that is behind NASA, the solar eclipse with NASA, the USGS store, and NOAA Fisheries, all of which are Drupal sites. Similar to these organizations, Drupal is the CMS system for you if your needs are more complex, you are developing an e-commerce portal, or if you have a large amount of content to maintain. If you have a Drupal project in the works or are about to migrate versions or CMS systems, Mobomo has some of the best and brightest Drupal developers in the DC area.Key Terms:
- Permissions – This is a tool for controlling access to content creation, modification, and site administration at the application level.
- Administrators can assign permissions to roles, then assign users to these roles.
- The first user receives all permissions.
- Template – This refers to a file to express presentation
- These are mostly written with a markup language that has variables representing data provided to the template.
- Theme Engine – This is a set of scripts that interprets code and makes theming a site easier. These scripts take the dynamically generated content and output it to HTML.
- The default theme engine is PHPTemplate
- Theme Hook – This is an identifier used by the calls to the theme() function to delegate rendering to a theme function or theme template. Modules which extend Drupal may declare their own theme hooks to allow editors to control the markup of that module in their theme.
- Trigger – These typically result from a characteristic change in an entity maintained by a module.
- Deleting content
- Adding a comment that a user has logged in
- Adding a term
- Triage – A new issue is assigned a priority based on its severity, frequency, risk, and other predetermined factors.
- Zebra Striping – This refers to the to the alternating colors between rows of data. It is most common for rows of data to alternate background colors between white and gray.
- Testbot – A continuous integration service for testing patches submitted to project issues on Drupal.org.
- Roles – A name for a group of users, to whom you can collectively assign permissions. There are two predetermined, locked roles for every new Drupal installation:
- Authenticated User – anyone with an account on the site.
- Anonymous User – those who haven’t yet created accounts or are not logged in.
- Additional roles can be added and users can belong to multiple roles.
- Path – This is the final portion of the URL that refers to a specific function or a piece of content.
Please reference Drupal.org for more information!
The post Key Drupal Taxonomy: Part 3 appeared first on .
This blog has been re-posted and edited with permission from Dries Buytaert's blog. Please leave your comments on the original post.
In the beginning of the year I started doing some iOS development for my POSSE plan. As I was new to iOS development, I decided to teach myself by watching short, instructional videos. Different people learn in different ways, but for me, videos tutorials were the most effective way to learn.
Given that recent experience, I'm very excited to share that all of the task tutorials in the Drupal 8 User Guide are now accompanied by video tutorials. These videos are embedded directly into every user guide page on Drupal.org. You can see an example on the "Editing with the in-place editor" page.
These videos provide a great introduction to installing, administering, site building and maintaining the content of a Drupal-based website — all important skills for a new Drupalist to learn. Supplementing user guides with video tutorials is an important step towards improving our evaluator experience, as video can often convey a lot more than text.
Creating high-quality videos is hard and time-consuming work. Over the course of six months, the team at Drupalize.Me has generously contributed a total of 52 videos! I want to give a special shout-out to Joe Shindelar and the Drupalize.Me team for creating these videos and to Jennifer Hodgdon and Neil Drumm (Drupal Association) for helping to get each video posted on Drupal.org.
What a fantastic gift to the community!
Retrospectives are an essential part of our team’s workflow. After each iteration, we get together to collect insights and feedback. By doing so, our teams ensure they have time to celebrate achievements, learn from mistakes and steer their efforts along a process of continuous improvement.Josef Dabernig Tue, 05/29/2018 - 17:40
What are the steps of a retrospective?
Retrospectives will often be made up of 3 simple steps: a) What went well? b) What could we have done better? c) Action items for further improvements. More in-depth retrospectives can use the following model for deeper analysis:
1) Set the stage
A brief check-in allows everyone to get ready for the retrospective, i.e. we gauge how everybody is feeling about the past iteration.
2) Gather data - What?
The data gathering stage is all about collecting different viewpoints based on the metrics of how the sprint went, external feedback the team has received or things they have observed during the iteration. For retrospectives of longer time periods, we use a timeline to collect major milestones from participants and discuss them in a group.
3) Generate insights - So What?
Here we go into problem solving mode. Using brainstorming activities we are able to determine the reasons why things went well or not. For example, the 5 Whys can be used to identify root causes or by imagining The Worst We Could Do, our teams find out what they need to improve on.
4) Decide what to do - Now What?
Now it’s time for the team to create actions that will help them to become even better in the next iteration. Practices like Circle of Influence helps to focus them on what they can accomplish as a team. We find Divide the Dollar to be useful as well as other dot-voting activities when determining what we want to focus on.
5) The closing perspective
Finally, in the closing, we want to make sure that everyone gives their final input on how the retrospective went.
Things to keep in mind when running retrospectives
Retrospectives done right are a powerful tool to help your team open up and have meaningful conversations. As with any meeting, it’s important to ensure everybody is on board with the working arrangements, such as being on time and a willingness to contribute. As the facilitator of the meeting, you can do a great job at providing a space where participants feel encouraged to share what’s really on their mind.
Looking for ways to make your retrospectives more engaging? Retromat is a tool that helps you think of different ways to facilitate a retrospective. In terms of online collaboration, we found meeting on zoom.us with Realtime Board and collaborating on our retrospective notes in a shared Google Slides presentation to be most effective.
Thanks for reading our take on retrospectives. If you'd like to learn more about running retrospectives effectively, don’t hesitate to reach out in the comments section or get in touch using our contact form.
Did you know that Drupal has a Point of Sale (POS) module that pairs with the widely used Commerce module? That's right, Drupal Commerce is now the full end-to-end platform for a complete omnichannel ecommerce experience. Whether you're running an online store, a physical store, or both, you can do it all with Drupal Commerce!
One of the great things about a web-based POS is that all you need is a web browser for it to work. This opens the door to new POS hardware options. You can use an iPad, a laptop, or anything that has a browser. You don't need any expensive or specialized hardware from Moneris, nor do you need a branded solution such as Square. Instead, you now even have the option to build your own POS hardware for very little cost. Today we're featuring a Raspberry Pi based prototype that WE built! The whole setup cost about $250 CAD.
Watch the video below, or keep reading to learn more.
As mentioned above, we bought a simple touchscreen and mounted a Raspberry Pi on the back. Once up and running, all you have to do is plug it in, connect it to the Internet, and it will automatically boot up into the POS login screen. If your staff has a problem, all they have to do is unplug it and plug it back in. There's no messing with settings or anything. Just reboot. Easy!
- The administrative view, which is what the cashier would use.
- A customer display view, which shows what the cashier has added so the customer can see the products and prices entered in real-time. Remember: all you need is a browser and something that can display a browser. The customer display is especially easy because it doesn't have to be a touchscreen; you could just use any monitor, a TV, etc, and run it off of the cashier hardware.
- A kiosk view, which is basically just running the front end of the site like your customers would do on their home computers. You could set that out in your store and let customers browse products and make purchases.
So, for a shoestring budget, we created a working point of sale that could be used in a store (see the video above). Aside from looking a little silly, our example is perfectly fine and works great. Plus, there are endless options for inexpensive enclosures to make it look better. You could even build or 3D print your own.
The do-it-yourself (DIY) route is a lot cheaper and gives you the freedom to do whatever you want. We will post further details soon on how to do all this yourself, including specific links to the components we used. And remember: it's Drupal, so it's open source, and all the software is free.