- Datastore Publish
Explore how to extend scalable Cloud Dataflow pipelines to publish machine learning predictions to both BigQuery and Cloud Datastore. Understand the process of writing entities to Datastore for low-latency querying and the advantages of batch operations. This lesson helps you build pipelines that serve analytical and application databases, enabling product teams to access personalized prediction data efficiently.
We'll cover the following...
Publishing results to BigQuery is useful for ETLs and other applications that are part of batch data pipelines. However, it doesn’t work well for use cases where applications need to retrieve a prediction for users with low latency.
Cloud Datastore
GCP provides two NoSQL databases that provide a solution for this use case, in which you need to retrieve information for a specific user with minimal latency.
In this lesson, we’ll explore Cloud Datastore, which provides some querying capabilities for an application database. We’ll build upon our prior pipeline and add an additional step that publishes to Datastore while also publishing the predictions to BigQuery. ...