The Ongoing Journey Within: Discovering Your Path to Self-Improvement

Have you ever paused to reflect on who you are today and who you aspire to be tomorrow? The path of self-improvement is a deeply personal one, filled with moments of insight and growth. Along my own journey, I've encountered principles that have profoundly shaped my career and daily life. I hope these reflections offer you some guidance on your own path:

1. The Architect of Your Choices: Take a moment to acknowledge the power of your decisions. If you find contentment in your present, celebrate the good choices that led you here. If not, view it as an opportunity to learn and consciously steer your future with wiser choices. * Insight: Observe the people around you whom you perceive as successful. Consider the qualities they possess that resonate with you. Remember, their journey is unique, but identifying admirable traits can offer valuable direction.

2. The Company You Keep, The Person You Become: A dear friend once shared a powerful truth: "If you are alone and feeling miserable, you are in bad company." Our environment significantly influences our well-being and growth. * Insight: You are drawn to others for a reason – either they mirror who you are, or they represent who you wish to become. Lean towards the latter. Spend time understanding what attracts you to them and consciously work towards embodying those positive qualities.

3. Nurturing Your Inner Worth: Self-esteem is the quiet confidence that fuels our actions. Without it, we may find ourselves plagued by feelings of inadequacy and a multitude of reasons not to try. * Insight: True self-respect blossoms from the discipline of completion. Whether it's achieving a fitness goal, mastering a new skill, finishing a task, or keeping a promise, each accomplishment reinforces your sense of self-worth.

4. The Beauty of Incremental Growth: The journey of self-improvement can feel overwhelming at times. Remember that progress, in any measure, is a step in the right direction. * Insight: Break down ambitious goals into smaller, more manageable steps. Focus your energy on the present task. The timeless wisdom, "The journey of a thousand miles begins with a single step," reminds us that even the grandest achievements start with a small act.

5. The Rhythm of Growth: Routines with Purpose: Our daily routines shape our lives. A balanced routine should address both our immediate responsibilities and our long-term aspirations. * Insight: Think of your day-to-day tasks as "Survival Mode." While necessary, solely operating in this mode leaves little room for growth. If your daily tasks consume all your time, it's a signal to enhance your efficiency and acquire skills that can streamline your current work, freeing up energy for future-focused activities.

6. The Power of Single Focus: Spreading your energy across too many endeavors can lead to stress and diluted results. * Insight: At any given time, identify one primary goal in your personal, team, and organizational spheres. Concentrating your efforts allows you to maximize your potential. These focal points can evolve quarterly as you achieve your objectives.

7. Embracing the Unfamiliar: The Catalyst for Confidence: I've often found myself drawn to new challenges, even when they feel uncomfortable. This willingness to step outside my comfort zone has been a powerful confidence builder. * Insight: When your current work becomes effortless and requires little conscious thought, it might indicate a plateau in your learning. Seek out new tasks and projects that challenge you and spark new ways of thinking.

8. Navigating the Edge: The Fear of Loss: The fear of losing can be a double-edged sword, either fueling our efforts or creating undue pressure. * Insight: Understand your personal response to the fear of loss. Be aware of its influence and strive to harness its motivating potential without letting it become detrimental.

9. The Unwavering Foundation: Health is Wealth: This isn't just a cliché; it's a fundamental principle. Your health – both physical and mental – is the cornerstone of all your aspirations. * Insight: In our digitally connected world, resources on health and well-being are abundant. Prioritize your health as the primary ingredient for a successful and fulfilling life.

10. The Art of Connection: The Importance of Soft Skills: Interestingly, those who excel in their careers often do so because of their ability to connect with and understand others. Technical skills are valuable, but soft skills are often the differentiator. * Insight: Value your mentors. They offer a crucial external perspective that can illuminate blind spots and guide your growth. Your own viewpoint is just the starting point; the insights of a trusted mentor can often reveal deeper truths.

11. Finding Freedom in Failure: A significant lesson I've learned is the importance of detaching your identity from failure. When efforts don't yield the desired outcome, it's an opportunity to learn and adapt. * Insight: Instead of blaming yourself, analyze what went wrong. Develop a plan for improvement and try again. Attributing failure to your inherent worth is a disempowering mistake.

12. The Clarity of Purpose: Understanding the What, Why, and How: Before embarking on any endeavor, take the time to understand its essence (WHAT), its purpose (WHY), and your approach (HOW). This clarity streamlines your efforts and helps you eliminate unnecessary actions. * Insight: Regularly ask yourself these three fundamental questions to ensure your actions are aligned with your goals and values.

13. The Unexpected Gift: Opportunity in Crisis: Some of the most significant opportunities for growth have emerged from challenging times. * Insight: Cultivate a mindset of a problem-solver. Instead of dwelling on difficulties, seek solutions and identify the potential for growth within the crisis.

14. Reawakening the Inner Learner: As children, we possess an innate curiosity and an uninhibited ability to learn. Somewhere along the way, judgment can stifle this natural gift. * Insight: When someone offers feedback, approach it with an open mind. Introspect before dismissing it. Often, our own preconceived notions are the biggest barriers to learning.

15. The Quiet Strength Within: Believing in Yourself: Every human being possesses unique talents and gifts. Recognize and nurture yours. * Insight: Reflect on moments in your life when you felt truly successful and capable. Acknowledge your strengths and consciously build upon them as you navigate your future.

16. The Ripple Effect: The Power of Appreciation: Take the time to appreciate your own efforts and acknowledge the contributions of others. Self-appreciation fosters self-awareness, while appreciating others cultivates humility and a willingness to learn. * Insight: Make appreciation a regular practice in your interactions with yourself and others.

17. Leaving Your Mark: The Essence of Impact: To create a lasting legacy, simply fulfilling tasks isn't enough. Strive to make a meaningful impact in all that you do. * Insight: Aim to look back on your work with pride. This requires a deeper level of engagement and a commitment to making a difference.

The Heart of the Matter: The journey of self-improvement is a continuous evolution. If your current path resonates, continue. If not, be open to trying new approaches. Avoid repeating mistakes in the pursuit of success. In our fast-paced lives, mastering efficiency is key to achieving more. Keep exploring, keep reflecting, and keep growing into the best version of yourself.

1:1 what it IS and what it is NOT

Let me start with what it is NOT

• It is not a meeting for status check for on going activities. There are other venues to get that
• It is not about others - we might briefly touch point but the core focus is on individual

What IS 1-1 about

• Opportunity for Managers to understand career goals of individuals and areas of interest they want to work on
• Managers to find ways to help individuals achieve personal / technical goals (with in the context) while achieving business goals
• Managers to help the individuals to grow personally and professionally
• Managers to suggest alternative ways based on their past experience
• Managers observe how you do your day to day task and have their inputs ready during the meeting
• Managers to identify areas of improvement and come up with action items, so that they can followup in the next 1-1
• Managers to keep measuring and suggest corrective actions for a smooth course of achieving the objectives
• Managers to seek feedback from individual what is working and not working on the floor
• Individual point of view try to gather data for the above to speak to your manager
• Managers could be a Mentor or a Coach (Mentoring is a long-term process based on mutual trust and respect. Coaching, on the other hand, is for a short period of time on a skill etc.)
• Constructive criticism / feedback is the base of conversation

Highlights

• Personal touch - Managers should personally invest time in the individual to come up with suggestions
• Identify - skill gaps, mistakes observed during an instance, participation in activities
• Action items - to work upon
• Appreciate key results

Tools to help the process

• Objectives and Key Results - https://www.youtube.com/watch?v=L4N1q4RNi9I
• Checklist of goals - quarterly, half yearly and yearly

Reasons mostly employee quit

• They quit on their managers not the company in most occasions. When employee feels they are not taken care or being heard, they get frustrated and eventually leave

End Game:

• If an individual is willing to join his Manager in another team or company is measure of the Manager's success and being impactful on individual's career
• You might agree, we still stay in touch with our effective Managers, who influenced us and our careers at one point

Apache Spark Overview - Part I

Apache Spark - manages and coordinates the execution of tasks of data across a cluster of computers

Cluster Manager:

• The cluster of machines Spark uses to execute the tasks managed by Cluster Manager
• When we submit Spark application to Cluster Manager, they allocate resources for our applications
• Spark application consists of Driver + Set of Executor Processor
• 3 types of cluster managers - Standalone, Yarn and Mesos

Driver:

• Driver runs main() function and sits in one of the nodes in cluster
• Driver does (1) Maintain information about Spark Application (2) Respond to program or input (3) For Analyzing, Distributing and Scheduling work
• Drivers assign work to Executors

Data:

• Data Sets which nothing but typed data
• Data Frame can span multiple computers for one simple reason as the data is large. it is untyped.
• Resilient Distributed Data (RDD)

Transformation:

• Core data structures of Spark are immutable (cannot be changed after created)
• Instruction to change data is called transformation
• Types of Transformation - Narrow (one input - one output) and Wide (one input - many outputs)

Action:

• To trigger transformation computation we need action
• 3 kinds of actions - (1) View data in console  (2) to collect in native objects of languages (3) write output to data sources

Spark UI:

That shows state of your job environment and cluster state

Key aspects of observability for a Platform

Isolate Traces, Logs, Metrics so that we can focus on right problem

Traceability

1. Ability to trace origin of the calls - User / IP etc.
2. Request end to end traceability with unique identifier
3. Ability to measure duration of execution in each hop
4. One destination / system to capture and query for end to end visibility
5. Ability to persist data for agreed duration
6. One view to track health of your system by only showing the failures or slow responsive system

Logs

1. Isolate errors / exceptions
2. Ability to control debug / info / error levels
3. Avoid too much information
4. Understandable by anyone outside your team
5. Right categorization so that you can focus on finite data

Metrics

1. Ability to detect failures like HTTP 5xx etc.
2. Ability to detect anomalies in a given duration
3. Ability to be able to view and navigate through - Application and Infrastructure
4. If possible build a drill down
5. Don't forget - every level in the organization pyramid wants a different view of these metrics

One key aspect for validating completeness - ask support team or someone outside your team to validate the completeness of steps and documentation for triaging. 

Hmmm Enterprise Patterns tiring but useful

Request-Reply

Consumer / Producer with Request - Reply Channel
Multiple Consumers / single producer with Request-Reply Channel

Return Address
Consumer specifying return address in the request message. Producer sends the response to the return address
Request getting consumed by more than one service provider

Correlation Identifier
Each message is having Message Id, CorrelationId, Business Id (ex: orderId)
Provider copies the these id's to the replies

Pipes and Filters
There is always queue / pipe / channel between filters i.e. the sequence of execution

Context-based routers
Based on context routed to multiple service providers and output channels

Splitter
Splitting the messages into individual message consisting data for one item

Splitter-Router
This is combination of splitter and context-based routing

Aggregator
Multiple services responses are combined together by aggregator (example: for completing order we may have to do multiple tasks, aggregate the response and send the response)

Scatter-Gather
Example: Sending quote to multiple lenders and sending final quote that is with lowest rate of interst

Composing Patterns
Combination of Splitter -  Scatter-Gather 

Redis capabilities - I'm impressed

1. Optimized for speed - Single threaded event loop
2. Data structures are implement for maximum performance, constant time and space
3. Binary safe - human readable text to encoded binaries
4. Size - 0 bytes to 0.5 GB
5. Key based access - using GET and SET semantics
6. Multi key operations
7. Atomicity of operations
8. Data expiration - TTL (Time to Live) support. These keys are called volatile keys (Advantages: effective invalidation approach and keeping cache size under control)
9. Configurable eviction policies (all or only volatile keys) - Supports random or usage based eviction policies
10. Intelligent caching - can managed via commands on server or LUA scripts
11. Distributed network server - uses TCP protocol
12. Request pipeline - i.e. batch process to boost average request per pipeline
13. Master - Slave architecture
14. Highly available - Replication mechanism through slave. Watchdog and management processes failover when needed
15. Persisting data to durable storage - supports optimal and tunable data persistent mechanisms. Also can create snapshots of cache at any moment in time and restore to instant before outage
16. Horizontal scaling support for RAM, Computation or Network resources via Sharing
17. Local cache replicas - can be run on application servers and can be used for local caching. Replication can be used for local replicas for central shared cache.
18. RLEC - Redis Lab Enterprise Cluster acts as container that can run multiple redis databases in a highly available and scalable manner.
19. Zero touch service deployment - cloud (AWS, GCP, Azure, IBM) based deployment
20. https://redislabs.com/redis-enterprise/technology/redis-enterprise-cluster-architecture/
21. https://redislabs.com/redis-enterprise/technology/redis-enterprise-cluster-architecture/

AWS Architecture and Design Principles

Software Design Principles

• Be clear for what you are designing -
• What is the production traffic 1x
• Who is going to access
• How personas are going access
• How can we troubleshoot
• Are you able to trace everything that happened 
• Make sure to test your production systems at different scales 1x to 10x
• Test the fail over scenarios
• Test the failure scenarios - when dependency system / service not available etc.
• Have alerts in place for failures and anomaly detection
• Pick the right technology for use case

Tricks: AWS SNS retry

If you have a service with async processing and you would like to setup retries - instead of building your own solution please take a look at https://docs.aws.amazon.com/sns/latest/dg/DeliveryPolicies.html

You can control the frequency of the retries in the backoff phase with three parameters.

·       Minimum delay—The minimum delay defines the delay associated with the first retry attempt in the backoff phase.

·       Maximum delay—The maximum delay defines the delay associated with the final retry attempt in the backoff phase.

·       Retry backoff function—The retry backoff function defines the algorithm that Amazon SNS uses to calculate the delays associated with all of the retry attempts between the first and last retries in the backoff phase.

You can choose from four retry backoff functions.

·       Linear

·       Arithmetic

·       Geometric

·       Exponential

Good luck