Upgrade from D6 in 7 steps? Hell no!!!

http://dancingczars.files.wordpress.com/2010/06/hellno.jpg
Sooo ... I've finally upgraded my own site. I thought it would be easy. A simple drupal site only using images. And yes I'm not reading READMEs that often. I trust drush somehow to handle this for me. But drush only sooth the 'pain'. This blog contains my bash script which does some staging from production onto my laptop where I update the site to the latest D6 then upgrade to D7. The script is generic enough to adapt for other upgrades.

Drush aliases

I use drush aliases so these are my aliases. The list is generated by drush sa build2be.com. The id dev is a similar environment as my (now old) prd. The d7 is my local upgrade environment. My file location is ~/.drush/build2be.com.aliases.drushrc.php
$aliases['build2be.com.dev'] = array (
  'uri' => 'http://build2be.com.dev',
  'root' => '/Users/clemens/Sites/build2be.com.dev/www/',
  'path-aliases' => 
  array (
    '%dump-dir' => '/Users/clemens/Sites/build2be.com.dev/store',
  ),
);
$aliases['build2be.com.d7'] = array (
  'uri' => 'http://d7.build2be.dev',
  'root' => '/Users/clemens/Sites/d7.build2be.dev/www/',
);
$aliases['build2be.com.prd'] = array (
  'uri' => 'http://build2be.com',
  'root' => '/home/b2bprd/sites/build2be.com',
  'remote-host' => 'server.com',
  'remote-user' => 'remote-user',
  'path-aliases' => 
  array (
    '%dump-dir' => '/home/b2bprd/store',
    '%drush-script' => '/home/b2bprd/.drush/drush/drush',
  ),
);

Preparation

The code below are from one bash script cut into pieces. This script can download a D6 production site onto your laptop by using DS as an argument. Next you can UPD the D6 site. As a last step you can upgrade the site to a D7 version by giving the UPG
#!/usr/bin/bash
set +x
#set -x

# DS: down stream a ; UPD: update ; UPG: upgrade ; BACKUP
TRANSITION=$1

[ "$TRANSITION" = '' ] && TRANSITION=HELP

# work with a cached version if available
#MODE=dev
# grab fresh content instead a cached version
MODE=prd

# How to open URLs
# - mac: open, 
# - windows: start(?)
# - linux: see http://stackoverflow.com/questions/3124556/clean-way-to-launch-the-web-browser-from-shell-script
URL_COMMAND=open

[ ${MODE} = 'dev' ] && DRUSH_CACHE=
[ ${MODE} = 'prd' ] && DRUSH_CACHE=--no-cache

# We assume build2be.com.aliases.drushrc.php exists
SA_GROUP=build2be.com

# we assume aliases for prd / dev / d7
SA_PRD=${SA_GROUP}.prd
SA_DEV=${SA_GROUP}.dev
SA_D7=${SA_GROUP}.d7

Helper function

I needed some helper function for showing help, showing instructions, getting the drupal root dir, launch an url, reset user 1 login, etc.
function help {
  echo We needs at least 1 argument.
  echo
  echo " DS  : bring system over from PRD to DEV aka downstream"
  echo " UPD : update the site to the latest version"
  echo " UPG : upgrade DEV into D7"
  echo
}

function get_root_dir {
  ROOT=`drush @${1} status | grep "Drupal root" | cut -d : -f2`
  echo $ROOT
}

# get_root_dir $SA_D7

function header {
  echo
  echo ==================================================================
  echo $1
  echo ------------------------------------------------------------------
}

function header_url {
  header "$1"
  echo Starting "$2"
  echo
  echo ... press enter to continu
  echo
  $URL_COMMAND $2
  read
}

function get_url {
  ROOT=`drush sa @${1} | grep uri | sed "s/.*http/http/" | sed "s/',//"`
  URL=${ROOT}/${2}
  header_url "$3" $URL
}

function uli {
  URL=`drush @${1} uli`
  header_url "$2" $URL
}

Downstream

So all function are in place. Now we can DS, UPD and/or UPG We grab both drupal root and it's database onto laptop.
function downstream {
  header "Downloading from PRD to DEV"
  # Grab a copy from prd into dev
  drush --yes $DRUSH_CACHE rsync @${SA_PRD} @${SA_DEV}
  drush --yes @${SA_DEV} sql-drop
  drush --yes $DRUSH_CACHE sql-sync @${SA_PRD} @${SA_DEV}

  get_url ${SA_DEV} logout "We logout to get in properly."

  uli $SA_DEV "Check DEV for proper operation after the downstream fetch (DS)"
}

Update to latest D6

Next update the D6 install to the latest versions. I'm not sure how the core upgrade went. Repeating should lie in zapping the dev environment completely first.
function update {
  header "Updating DEV"
  drush --yes @${SA_DEV} pm-update

  get_url ${SA_DEV} logout "We logout to get in properly."

  uli ${SA_DEV} "Check DEV for a successfull update (UPD)"

}

Upgrade to D7

The initial d7 install was done through drush @build2be.com.d7 site-install because drush site-upgrade failed on earlier attempts. I'll come back to this when having done another site upgrade using drush-5.
function upgrade {
  header "Disable modules before upgrading"
  drush --yes @${SA_DEV} pm-disable captcha
  # drush --yes @${SA_DEV} pm-uninstall

  get_url $SA_DEV logout "Make sure session gets killed before copying it over"

  header "Dropping old table remnants."
  drush --yes @${SA_D7} sql-drop

  header "Syncing files and database from DEV to D7."
  drush --yes $DRUSH_CACHE rsync @${SA_DEV}:%files @${SA_D7}:%files
  drush --yes $DRUSH_CACHE sql-sync @${SA_DEV} @${SA_D7}

  # this fails so we need to do it through the front-end
  # drush --yes @${SA_D7} pm-update
  echo
  echo ====================
  echo = As drush cannot bootstrap the site we need to manually visit the update.php
  echo =
  echo = Make sure you enable free access in settings.php
  echo =
  echo = Running this from a unstable site needs a mysql table drop
  echo = drush sql-drop might not work properly enough for this
  echo =
  echo ====================
  get_url $SA_D7 update.php "Run update.php manually."

  header "We need content_migrate and other modules"
  drush --yes @${SA_D7} dl cck-7.x-2.x codefilter

  header "Enabling modules"
  drush --yes @${SA_D7} en content_migrate codefilter image

  # header "Migrating all fields"
  # drush --yes @${SA_D7} content-migrate-fields

  uli ${SA_D7} "Check your site for proper operation"

  get_url ${SA_D7} admin/config/regional "Check Location and Timezone"

  header "Migrate fields"
  get_url $SA_D7 admin/structure/content_migrate "Migrate fields"

  drush --yes @${SA_D7} pm-disable content_migrate

  get_url $SA_D7 admin/config/content/formats/1 "Enable code filter format ... set weight after tag filter ... add 'p'"

  get_url $SA_D7 admin/structure/types/manage/blog/display "Fix the display field order and labels"

  get_url $SA_D7 admin/structure/types/manage/blog/display/teaser "Fix the teaser field order, labels and image"

  #drush --yes @${SA_D7} cc all

  get_url $SA_D7 node/117/edit "Save to to reanimate aliases."
  #drush @${SA_D7} vget path_alias_whitelist

  get_url $SA_D7 node/117 "Check the php and code display."
}

Execute

[ $TRANSITION = 'HELP' ] && help
[ $TRANSITION = 'UPD' ] && update
[ $TRANSITION = 'DS' ] && downstream
[ $TRANSITION = 'UPG' ] && upgrade
set +x

Left overs

Some of the issues below are covered in my script.
  • aliases where not active. http://drupal.org/node/1218954 Workaround: edit just a node then save.
  • pathauto tokens where not replaced (was it because the module was disabled?)
  • there was no theme selected (visiting the appearance page showed an error). Enable a theme.
  • break is not working. See http://drupal.org/node/881006
  • overlay not enabled. See steps from http://www.group42.ca/drupal_7_upgrades_scripts_drush
  • error on node edit. See http://drupal.org/node/952970 Workaround: visit all content types using taxonomy. Then edit/save. See also http://drupal.org/node/931512 for required node body fields.
  • images miss their extensions png gif jpg jpeg
  • The style directory is not creatable / writable. See http://drupal.org/node/873838

Conclusion

Writing a script over a document is invaluable when the upgrade quirks are huge. For this small site it was worth the effort as my time slots were small. After a week the steps to take were documented so I could fix the next step.

References

- http://drupal.org/project/drush - http://drush.ws - http://www.group42.ca/drupal_7_upgrades_scripts_drush - http://stackoverflow.com/questions/3124556/clean-way-to-launch-the-web-browser-from-shell-script - http://dancingczars.files.wordpress.com/2010/06/hellno.jpg