Q: How to explode and select normal and nested columns in pyspark? I have a table in Bigquery, and its schema I have uploaded as Image.
Schema
I am trying to make some transformation in spark using pySpark.
For my transformation,I need to select content.id( STRUCT) ,content.links ( STRUCT),
,content.authors  (RECORD REPEATED(i;e, array of STRUCTS))
But,I am not able to select them in a single statement.
Can anyone show me how I can achieve it using PySpark and SparkSQL?
Here is a dummy table and expected output:
expected output
