Search This Blog

Wednesday, July 22, 2020

How to Exporting the data from postgres


This example uses a single table, but a more complex result is possible as well, where several tables are joined. Here is an example script:

COPY (
SELECT
userid AS "userId.auto()",
'some-static-string' AS "__classname__.auto()",
failed_login_count AS "failedLoginCount.int32()",
CASE WHEN mail_validated=TRUE THEN 'TRUE' ELSE 'FALSE' END AS "mailValidated.boolean()",
-42 AS "answers.toTheQuestion.int32()",
'Slate Star Codex' AS "answers.niceBlog.auto()"
FROM
users.accounts
) TO '/tmp/users-accounts.csv' WITH (FORMAT CSV, HEADER TRUE, DELIMITER E'\t');




And now for the import, which — thanks to our preparations — is a breeze:

mongoimport -d db -c users --type tsv --file export.tsv --headerline --columnsHaveTypes


Conclusion

Migrating data from postgresql — or any other sql server — is easy enough and very performant with native commandline tools. If you do need to have more control over the data format, you obviously will run into problems soon enough, where a migration script based on python or nodejs might be better suited than a big one-liner.

No comments:

Post a Comment