I'm writing a new big book on software performance and scalability
If you'd like to see early chapter drafts from it, and potentially give me a little feedback (or answer a few simple questions I ask of you, about your impressions), in private? please let me know
For all the students out there: I used #trigonometry and queue #DataStructures to solve two different real world problems at work today. Don’t forget the basics!
I used it when I was busy with my #datastructures course. All of the #JetBrains software has advanced debugging tools.
So I used #PyCharm for #Python, #CLion for C++ and #InteliJ for #Java. It allowed me to debug connections and relationships in data structures.
Today, I wouldn't personally use it because I know those stuff now, and I have my own workflow now. I never step through code anymore, I don't even use the debugging in #VScode anymore.
When required, I am a degenerate "console.log" debugger.
4-min interview w/ Tarjan on his inverse-Ackermann upper bound for the union-find data structure . Turns out he got a matching lower bound on that algorithm first - I didn't know about the lower bound before! Interesting story behind the path of research.
any #python reading recommendations to optimize memory use of a dict containing lists which in turn contain lots of small dicts? once build up they can be read-only.
Reduce nesting: Avoid unnecessary layers of nesting. Flatten the structure or use simpler data types like tuples when inner dictionaries have few keys.
Example: Instead of List[Dict[Any, List[Dict[Any, Any]]]], consider List[Tuple[Key, Value]].
Custom classes: Create specific classes for inner dictionaries with fixed keys to improve efficiency, readability, and maintainability.
Example: Instead of {'name': 'John', 'age': 30}, use a Person class with attributes name and age.
Use NumPy or Pandas: For numerical computations or tabular data, utilize NumPy arrays or Pandas DataFrames for faster operations.
Example: Convert a nested list into a NumPy array for mathematical operations.
Dictionaries with fixed keys: If inner dictionaries have constant keys, consider using named tuples or custom classes for better performance.
Example: Replace {'x': 1, 'y': 2} with a named tuple or class Point(x=1, y=2).
Appropriate data structures: Choose data structures based on access patterns. Use dictionaries when accessing elements by specific keys.
Example: Use a dictionary to store items with unique identifiers.
Generators or lazy evaluation: Use generators or lazy evaluation techniques to process data incrementally, saving memory and improving performance.
Example: Use a generator to read large files line-by-line.
Profiling: Before optimizing, profile your code to identify resource-intensive areas and focus on the most impactful improvements.
Example: Use Python's cProfile module to analyze code performance.