Data & Integrations
      Back to home
      1. Help Center
      2. Projects, Process & Support
      3. Data & Integrations

      Data Migrations & Integrations Briefs, Planning & Execution

      Must have process and guidelines for planning and briefing data migrations

      Data migrations are incredibly complex with massive risk to the client if they are not done properly.

      Before embarking on a Data migration the person managing the work must have at least 4 hours free to do the initial planning and design and 2-3 consecutive days available to manage through the implementation and testing.

      Process Overview

      1. Up to 3 Hour Data Brief with the Client
        Identify what data needs to be migrated and any data that needs to be cleaned
      2. Data Design and Sample test import
        Receive sample data, complete initial cleanse and Complete the Data brief document and plan; data design, data migration test plan and sample data import and testing 
      3. Quote Data Migration
        Provide quote for the full tasks required to migrate data
      4. Approve Quote
        Client approves quote for the migration
      5. Migrate Data
        Cut-over comms, migration, testing and final approval 

      Note if the outcome of the initial brief is that the data migration is very simple and can fit within the remaining points of the Data Brief task then the recommendation can be made by the Data Migration Expert to remove any areas not completed in the Brief document and then complete the migration within the remaining Data Brief points.

      Data Migration Brief

      A Data Brief should be created from the "{3CODE} | {Project} | Data Brief | working" Template within the client project or presales folder.

      This brief will aim to confirm:

      • Original Platform
      • New Platform
      • What needs to be Migrated:
        • Contacts
        • Companies
        • Deals
        • Tickets
        • Activities - including what type
        • Attachments
        • Object Associations - multiple or single object associations
        • Anything else, e.g. custom objects
      • The approximate number of records of each of the above
      • Migration timeline:
        • When Access will be provided to the original platform
        • When the team will stop using the original platform
        • When the team will start using the new platform

      Data Migration Planning & Design

      Where possible existing migration tools should be used.

      Existing Migration Tool Exists

      1. PM to get a sample of 100+ records of each object to be migrated
      2. Data Migration Specialist to run the initial Data Brief session and to review the sample data 
      3. Data Migration Specialist to create a test plan for testing the data migration, this should include what objects and data needs to be verified and should reflect the original brief
      4. Data Migration Specialist to do a sample migration on a developer platform:
        1. This should be done at least 2 weeks prior to cutover and include at least 100 records of each object to be migrated
      5. Internally validate the sample records migrated against the original platform records, ensure:
        1. Samples of all record types included in the Data Brief are migrated
        2. Check off each item from the data test plan for the sample records
        3. All data in the new platform reflects the original platform data
        4. That all associations are working as expected
      6. Capture the steps and any issues with the migration process and original data
      7. Have the client approve the sample migration
        1. Check off each item from the data test plan for the sample records
        2. All data in the new platform reflects the original platform data
        3. That all associations are working as expected
      8. Data Migration Specialist to provide a breakdown of tasks required for the full migration to the PM to include in the quote

        Migration Tool does not Exist

        1. PM to get a sample of 100+ records of each object to be migrated
        2. Data Migration Specialist to run the initial Data Brief session and to review the sample data 
        3. Data Migration Specialist to review the APIs of the Original and New platforms to ensure that the data specified within the brief can be extracted and imported
        4. If anything is not available in the migration stop and organise a session with Commercial and the Client to discuss a way forward
        5. Data Migration Specialist to Get access to the original platform including access to the API
        6. Data Migration Specialist to test the data received from the API to ensure that the data specified within the brief can be extracted and imported - this should be tested for each piece of data specified in the brief
        7. If anything is not available in the migration stop and organise a session with Commercial and the Client to discuss a way forward
        8. Data Migration Specialist to complete a detailed review of the current platform and to review each object page specified in the Data brief and note what data needs to be migrated
        9. Data Migration Specialist to identify 100x sample records for each object
        10. Data Migration Specialist to create a test plan for testing the data migration, this should include what objects and data needs to be verified and should reflect the original brief
        11. Data Migration Specialist to test migrate the 100 sample records (to be done at least 2 weeks prior to cutover) 
        12. Once the data is migrated, execute the test plan - verify the data migrated for each object ensuring that data in the original platform records matches the data in the new platform
        13. If not, the Data Migration Specialist to troubleshoot and fix (with the PM, CRM team and Client as required)
        14. If anything can not be fixed stop and organise a session with Commercial and the Client to discuss a way forward
        15. Have the client approve the sample migration
          1. Check off each item from the data test plan for the sample records
          2. All data in the new platform reflects the original platform data
          3. That all associations are working as expected
          4. Note that the client should have this sample data approved by at least 3x CRM users who are intimate with the current data
        1. Data Migration Specialist to provide a breakdown of tasks required for the full migration to the PM to include in the quote

        Quote Migration

        1. The Data Specialists to complete the design and provide a quote for the completion of the migration
        2. Ensure that all migration items from the brief are included in the quote: Cleaning, development required to sync/export data from the source platform, development required to sync/import the data to HubSpot
          1. Note that HubSpot only allows the manual import of Contacts, Companies, Deals, Tickets, Products, Custom Objects and Notes
          2. You must use the Engagements API to import activities (Emails, Tasks, Meetings and Calls)
          3. You must also use the API to migrate files, there are a number of ways that files can be migrated and these should be investigated as part of the quote 
        3. PM to raise the quote in HubSpot, check on the data migration date and time - dependent on the client preferences this may need be done on a weekend to minimise CRM access downtime for the client team - out of hours migrations will incur an additional cost of 5 points
        4. Client to approve the quote
        5. PM to coordinate the cut over time and date and schedule the Data Migration Specialist for the final migration

        Final Integration / Data Migration / Cut Over

        1. Ensure that the client has sent internal comms, has completed training and reconfirm cut over date
        2. Create a dashboard of legacy numbers (using text dashlet) vs. actual numbers (reports of live numbers from integration/migration)
        3. Ensure that everything is prepped and ready to go prior to migration
        4. On cutover, if possible stop access to the original platform
        5. Coordinate with the Data Migration Specialist to start the migration
        6. PM to send comms to the Project Team that the migration has started
        7. If there are any major issues that are going to risk meeting the migration completion deadline, define what options are available and speak with the Client to discuss how they would like to progress
        8. PM to send comms every 2-4 hours with a status update on the migration
        9. Once the data is migrated, execute the test plan
          1. Verify the data migrated for each object ensuring that data in the original platform records matches the data in the new platform and that all data has been migrated
          2. Note testing of this data should include record counts of the original and new platforms for migrated records by record type - if there is a mismatch this is a failed test and it will need to be rerun
        1. If there are any issues notify the client and work with the Data Migration Specialist to resolve
        2. If the issues are going to take more than an hour to resolve, speak with the Client to discuss how they would like to progress
        3. Once all issues are resolved and the test plan is successfully passed notify the client and get them to do a final verification, note this should include at least 3x CRM users who are intimate with the current data
        4. If there are any issues work with the Data Migration Specialist to resolve
        5. If the issues are going to take more than an hour to resolve, speak with the Client to discuss how they would like to progress
        6. Once all issues are resolved send an email to all parties confirming the completion of the migration and organise for the retirement of the original platform (recommend 1-2 weeks from a successful migration just in case new issues are identified)

        Note during the final migration If there are significant issues which would cause CRM access to be down longer than acceptable the client may need to revert back to the original CRM and the migration may need to be stopped and rescheduled. So it is critical that you are in constant communication with the client around any issues that arise and estimated resolution times.

        If a migration fails a deep dive of the issues and retest of migration must be done prior to the next migration.

        Best Practice Steps for Data Integrity, Verification and Validation

        1.   Test at each step of importation that data is correctly transformed when necessary. If                    feasible, duplicate the dataset to ensure data integrity while wrangling before the final                    migration is made.

        2.   Data verification can be achieved by creating "block lines". Highlight lines of rows                          systematically throughout the dataset, and validate that the lines match up after each column        transformation during wrangling.

        3.   Sampling should be strenuous throughout the wrangling process. Choosing appropriate test        cases (such as testing the start and end values of the dataset, boundary values of category          metrics, or closely related entries) will greatly aid in the wrangling process.

        4.   Determine what the "unique key" of your dataset is, when importing. Many tools such as              HubSpot will require a unique key when matching up one dataset to another, so it is                      important that careful consideration of what the unique key for each of your rows are, when          using your specific migration tools. Unique keys are an efficient way to do data validation              through importations.