𝐒𝐞𝐜𝐨𝐧𝐝 𝐫𝐨𝐮𝐧𝐝 𝐨𝐟 𝐂𝐚𝐩𝐠𝐞𝐦𝐢𝐧𝐢 𝐃𝐚𝐭𝐚 𝐄𝐧𝐠𝐢𝐧𝐞𝐞𝐫 𝐈𝐧𝐭𝐞𝐫𝐯𝐢𝐞𝐰 𝐐𝐮𝐞𝐬𝐭𝐢𝐨𝐧𝐬
:
:
:
1. Describe your work experience.
2. Provide a detailed explanation of a project, including the data sources, file formats, and methods for file reading.
3. Discuss the transformation techniques you have utilized, offering an example and explanation.
4. Explain the process of reading web API data in Spark, including detailed code explanation.
5. How do you convert lists into data frames?
6. What is the method for reading JSON files in Spark?
7. How do you handle complex data? When is it appropriate to use the "explode" function?
8. How do you determine the continuation of a process and identify necessary transformations for complex data?
9. What actions do you take if a Spark job fails? How do you troubleshoot and find a solution?
10. How do you address performance issues? Explain a scenario where a job is slow and how you would diagnose and resolve it.
11. Given a dataframe with a "department" column, explain how you would add a new employee to a department, specifying their salary and increment.
12. Explain the scenario for finding the highest salary using SQL.
13. If you have three data frames, write SQL queries to join them based on a common column.
14. When is it appropriate to use partitioning or bucketing in Spark? How do you determine when to use each technique? How do you assess cardinality?
15. How do you check for improper memory allocation?