How to optimize algorithms? (parallel processing and/or faster algorithms). Provide examples for both

Data Science Interview QuestionsCategory: Data ScienceHow to optimize algorithms? (parallel processing and/or faster algorithms). Provide examples for both
1 Answers
MockInterview Staff answered 6 years ago

“Premature optimization is the root of all evil”; Donald Knuth
Parallel processing: for instance in R with a single machine.
– doParallel and foreach package
– doParallel: parallel backend, will select n-cores of the machine
– for each: assign tasks for each core
– using Hadoop on a single node
– using Hadoop on multi-node
Faster algorithm:
– In computer science: Pareto principle; 90% of the execution time is spent executing 10% of the code
– Data structure: affect performance
– Caching: avoid unnecessary work
– Improve source code level
For instance: on early C compilers, WHILE(something) was slower than FOR(;;), because WHILE evaluated “something” and then had a conditional jump which tested if it was true while FOR had unconditional jump.
Source

Your Answer

10 + 19 =