AWS S3 commands

https://chatgpt.com/share/670e13c1-19b8-8010-9d4b-dd41158d8a79

Maths course for Machine Learning

https://ocw.mit.edu/courses/18-065-matrix-methods-in-data-analysis-signal-processing-and-machine-learning-spring-2018/download/

AWS - Access Key ID vs Secret Access Key

https://chatgpt.com/share/48336e83-2376-439d-acd3-79ea2fe5b7d6

The term "Access Key ID" is not called "Secret Access Key ID" because the Access Key ID itself is not considered a secret. Here's the reasoning:

1. Access Key ID: This is a public identifier for the credential. It is used to identify the user or service that is making requests to AWS. The Access Key ID is not secret, and it can be shared (for example, in logs) without compromising the security of your AWS resources.

2. Secret Access Key: This is a private component that must be kept confidential. It works like a password and is used in combination with the Access Key ID to sign API requests. Anyone with access to both the Secret Access Key and Access Key ID can make authorized API requests, so the Secret Access Key must be kept secret.

3. Session Token: In the case of temporary credentials from AWS STS (Security Token Service), the Session Token is also a secret and is used in conjunction with the Access Key ID and Secret Access Key to authenticate API requests for the session's duration.

Thus, the "Access Key ID" is the public-facing part that identifies the key, while the "Secret Access Key" and "Session Token" are sensitive, secret components. That's why the "Access Key ID" is named as it is—it's meant to be distinguishable from the actual secret that provides authentication.

AWS - Security Token Service (STS)

AWS STS (AWS Security Token Service) is a service that allows you to request temporary, limited-privilege credentials for users or services to access AWS resources. These credentials are often used in scenarios where you don't want to use long-term access keys, or when you need to provide short-term access to resources without creating permanent IAM users.

Here’s a breakdown of key concepts related to AWS STS:

Key Features of AWS STS:

1. Temporary Security Credentials: STS provides credentials that are valid for a limited duration (from a few minutes to several hours). After they expire, they cannot be used to access AWS resources.

2. Assume Roles: One of the most common uses of AWS STS is assuming roles. For example, you can allow a user or application to "assume" a role that gives it specific permissions, even if the user or app doesn't have those permissions by default. This is useful in cross-account scenarios, federated access, or when using roles to segregate access levels within the same account.

3. Federation: AWS STS enables identity federation, which allows users from external systems (like corporate directories or external identity providers) to access AWS resources without the need to create IAM users for them. This is done via standard protocols like SAML 2.0 or OpenID Connect.

4. Cross-Account Access: STS makes it easy to provide access to resources across different AWS accounts. A user in one account can assume a role in another account and gain access to resources, all without requiring the user to have an IAM user in the other account.

5. Reduced Credential Exposure: Since STS tokens are temporary, they limit the risk if credentials are compromised. After the expiration time, the token cannot be reused.

Example Use Cases:

• Delegating Access: You have a central administrative account, and you want to allow users from other AWS accounts to access your account’s resources. They can assume a role that you define in your account to temporarily gain the necessary permissions.
• Third-Party Access: You can give third-party vendors temporary access to your resources by letting them assume a role through STS without needing long-term credentials.
• Mobile Apps and IoT Devices: Instead of embedding long-term credentials in mobile or IoT devices, which can be insecure, you use STS to issue temporary credentials, which can be refreshed as needed.

Example of Using STS with AssumeRole:

aws sts assume-role --role-arn arn:aws:iam::123456789012:role/example-role --role-session-name session1

This command allows an identity to assume the role example-role in account 123456789012 and gain temporary credentials.

In summary, AWS STS is a powerful tool for managing short-term, secure access to AWS resources without the need for long-term credentials. It’s especially valuable in complex environments that require secure, time-bound access control, such as in cross-account access or federated identity setups.

AWS IAM Identity Policy - "Resource" discussion

https://chatgpt.com/share/a15d63d2-a124-4e5d-8195-830e27fb395d

Whether targeting a role or specific AWS resources is more common or advised depends on the context and the desired security model. Both approaches serve different purposes in AWS Identity and Access Management (IAM), and each has its own use cases:

1. Targeting an IAM Role (More common for users/roles):

• Use Case: When you want to define a set of permissions for who can perform actions.

• Common: This is a common practice for controlling permissions at a higher level. You assign policies to roles, and these roles are then assumed by users or services.

• Advised: This approach is recommended when managing multiple users or services because you can easily modify or revoke permissions in one place (the role) without touching individual resources.

• Example:

  
  "Resource": "arn:aws:iam::123456789012:role/MyRole"
  

Advantages:

• Easier Management: You can centrally manage access by assigning or removing permissions from the role without having to modify each resource.

• Scalability: If multiple users or services need the same set of permissions, it's easier to manage by assigning roles rather than specific resource-level permissions.

When to Use:

• For multi-user or multi-service environments where permissions are broadly defined.

• For complex environments where you want to delegate access to external entities (cross-account access, etc.).

• When you need to limit actions based on who or what is trying to perform the action.

2. Targeting Specific AWS Resources (More common for resources):

• Use Case: When you want to limit what resources can be accessed or acted upon.

• Common: Resource-based policies are used when you need fine-grained control over specific AWS resources. This ensures that only designated users, roles, or services can access or modify those resources.

• Advised: Targeting specific resources is more advisable for high-security environments, or where sensitive data and actions need strict control.

• Example:

  
  "Resource": "arn:aws:s3:::my-bucket/my-object.txt"
  

Advantages:

• Granular Control: You can enforce permissions on specific resources, ensuring tighter security.

• Increased Security: You can prevent access to anything outside the explicitly defined resource, reducing the attack surface.

When to Use:

• When you need to apply specific permissions to individual resources, like a particular S3 bucket, EC2 instance, or Lambda function.

• When fine-grained permissions are necessary to ensure only specific actions are allowed on specific resources.

• In cases where resources belong to sensitive environments or need to be tightly controlled.

Which is More Common?

• IAM roles are generally more common in large-scale deployments and multi-service environments because they offer flexibility and ease of management.

• Resource-based targeting is frequently used for controlling access to specific, high-security or sensitive resources.

Which is More Advised?

• Targeting Roles is recommended when you want to manage permissions for who can perform actions (identity-based access control).

• Targeting Specific Resources is more advisable when securing specific resources in high-security contexts (resource-based access control).

Best Practice:

In many cases, it's beneficial to combine both:

• Use roles to manage who can access resources.

• Use resource-level permissions to restrict what those roles (or identities) can do on specific resources.

For example, an S3 bucket might have a policy that allows only a specific IAM role to perform actions, and that role has a policy that allows specific users to assume it.

Web Browser Event Loop

AWS Networking contd. - NACLs, Subnets, Security Groups

Discussion with ChatGPT about NACLs, Subnets, Security Groups

https://chatgpt.com/c/51ec2d12-d3ad-42e6-b20a-30470c0f442e

(Not a shared link. Have to be logged-in to chat.openai.com)

DSA - Leetcode - Some patterns

AWS Security - IAM Instance Profile, Alternatives for Credential Management, Trust Relationship Policy vs IAM Policy

https://chatgpt.com/share/06c43d08-9439-4493-b7ea-d000bde22c77

https://chatgpt.com/c/65373936-6c8d-4f70-94ab-c16276816eca

AWS Networking - Relationship between Region-VPC-AZ-Subnet-EC2

 

Amongst the following entities - Region, VPC, AZ, Subnet, EC2, which of these are associated with Routing Tables and NACLs?

Routing Tables and Network Access Control Lists (NACLs) are associated with the following entities in AWS:

1. **VPC (Virtual Private Cloud)**:

   - **Routing Tables**: Each VPC has one or more routing tables that control the routing of traffic between the subnets within the VPC and the internet, or other VPCs. These routing tables determine where network traffic from the subnet is directed.

   - **NACLs (Network Access Control Lists)**: NACLs are optional layers of security that act as firewalls for controlling traffic into and out of subnets. You can apply NACLs to each subnet within a VPC, and they provide stateless filtering of ingress and egress traffic at the subnet level.

2. **Subnet**:

   - While subnets themselves don't directly contain routing tables or NACLs, they are associated with them:

     - Each subnet is associated with a routing table from its VPC to manage the flow of traffic out of the subnet.

     - NACLs can be applied to subnets to manage access and traffic flow into and out of the subnet.

**Regions** and **Availability Zones (AZs)** do not directly involve routing tables or NACLs, as these are specifically related to network configurations within VPCs. **EC2 Instances** are affected by these configurations but do not themselves contain or directly manage routing tables or NACLs.

What about Security Groups? Which entity or entities are those associated with?

**Security Groups** in AWS are primarily associated with the following entities:

1. **EC2 Instances**:

   - Security Groups are used to control inbound and outbound traffic at the instance level. They act as a virtual firewall for EC2 instances to regulate allowable traffic.

2. **VPC (Virtual Private Cloud)**:

   - Security Groups are defined at the VPC level and can be associated with any instance in that VPC. While they are created within a VPC, their rules apply to instances they are attached to, not subnets.

Unlike NACLs, which are associated at the subnet level and provide stateless filtering, Security Groups provide stateful filtering and are associated directly with instances, such as EC2 instances, RDS instances, and Elastic Load Balancers within the VPC. This means they remember the state of traffic flows and can automatically allow return traffic without additional rules.

So all network security instruments - Security Groups, Routing Tables, NACLs are defined at the VPC level?

Yes, that's correct! All these network security instruments—Security Groups, Routing Tables, and Network Access Control Lists (NACLs)—are defined at the VPC level in AWS. Here's how each operates within the VPC:

1. **Security Groups**: These are associated with individual instances within the VPC, such as EC2 instances. They control inbound and outbound traffic at the instance level and are stateful, meaning they track the state of network connections.

2. **Routing Tables**: These define the rules for traffic routing within the VPC and between the VPC and other networks. Each subnet in a VPC is associated with a routing table that determines where network traffic is directed.

3. **NACLs (Network Access Control Lists)**: These provide a layer of security that acts as a firewall for controlling traffic into and out of subnets. NACLs are stateless, so they do not keep track of the state of network connections and each packet is processed individually.

All three instruments are configured at the VPC level but apply at different scopes—Security Groups at the instance level, Routing Tables at the subnet level, and NACLs at the subnet level—to provide comprehensive network security and routing management within a VPC.

Understanding usage of Database Indexes with Rails

Types of Database Indexes when using Rails:

https://chatgpt.com/share/1c4a94ef-2f5d-4c77-8f7c-ebded5e6bf3c

Indexes for Dates:

https://chatgpt.com/share/e71c3bd7-3a05-48d6-8064-7dd64fb53672

System Design - Kafka vs. RabbitMQ vs. Messaging Middleware vs. Pulsar; Http2; Idempotency

https://open.substack.com/pub/bytebytego/p/ep117-what-makes-http2-faster-than?r=1lyxr2&utm_campaign=post&utm_medium=email

Temporal courses

https://learn.temporal.io/courses/

Rama - competitor to Temporal

https://blog.redplanetlabs.com/2023/08/15/how-we-reduced-the-cost-of-building-twitter-at-twitter-scale-by-100x/

Workflow Engine Principles

https://temporal.io/blog/workflow-engine-principles

https://share.transistor.fm/s/65974dd6

Join Jason for another round of “Build Things on Purpose.” This time Jason is joined by Maxim Fateev and Samar Abbas, co-founders of Temporal, to talk about the software and solutions they are developing for orchestrating micro services. Maxim and Samar talk about their joint work in the past on various projects to include the Cadence project, which has laid the foundation for what they are continuing to do at Temporal. Check out this “Build Things” conversation as Maxim and Samar go into the details on the nature of their work at Temporal!

Durable Execution

https://orkes.io/blog/why-your-applications-need-durable-execution/

https://materializedview.io/p/durable-execution-justifying-the-bubble

First heard about it in a podcast featuring Temporal's CEO

System Design: 11 basic & essential concepts

Below is from a post by Kartikeya Sapra on LinkedIn

11 basic & essential concepts:

1. Scalability
- Ensures systems can handle increased load and grow efficiently.

Link → https://lnkd.in/dD-GZpVq

2. Latency vs Throughput
- Balances speed and capacity in system performance.

Link → https://lnkd.in/dscK9g3E

3. CAP Theorem
- Explains the trade-offs between consistency, availability, and partition tolerance in distributed systems.

Link → https://lnkd.in/dFpgDSnY

4. ACID Transactions
- Guarantees reliable database processing through atomicity, consistency, isolation, and durability.

Link → https://lnkd.in/dkkmMu_D

5. Rate Limiting
- Controls the amount of incoming and outgoing traffic to prevent overload and abuse.

Link → https://lnkd.in/dY9NqRG9

6. API Design
- Ensures APIs are user-friendly, efficient, and maintainable.

Link → https://lnkd.in/dTgxGa5i

7. Strong vs Eventual Consistency
- Balances the need for immediate data consistency versus eventual data accuracy in distributed systems.

Link → https://lnkd.in/dQDwa7TQ

8. Distributed Tracing
- Helps monitor and debug complex, distributed systems by tracking requests across multiple services.

Link → https://lnkd.in/dA-3swq2

9. Synchronous vs Asynchronous Communications
- Determines how systems interact, either waiting for responses (synchronous) or continuing independently (asynchronous).

Link → https://lnkd.in/dx2nFDgR

10. Batch Processing vs Stream Processing
- Differentiates between processing large volumes of data at once (batch) and processing data in real-time (stream).

Link → https://lnkd.in/dqGFQppV

11. Fault Tolerance
- Ensures systems remain operational despite failures or errors.

Link → https://lnkd.in/dzKWh4ju

Discussion about Cookies in Rails

https://chatgpt.com/share/0f6372ef-92d7-4719-9380-acbb428a21ab

MySQL CLI Output Formatting

https://chatgpt.com/share/4c875522-6581-4e91-97b0-1c786e65760e

mysql> pager less -S

Kamal

https://kamal-deploy.org/

Amazon Q

 

"Too many database connections" issue research

https://chat.openai.com/share/6dff5167-3924-4f8b-b0c2-97618d8653f6

Database Sequences

 

AI Book - Deep Learning for Coders with Fastai and PyTorch: AI Applications Without a PhD

Deep Learning for Coders with Fastai and PyTorch: AI Applications Without a PhD

https://course.fast.ai/Resources/book.html

testcontainers.com

https://testcontainers.com/

An open source framework for providing throwaway, lightweight instances of databases, message brokers, web browsers, or just about anything that can run in a Docker container.

DevOps Roadmap 2024

https://github.com/milanm/DevOps-Roadmap

LLM Course

https://github.com/mlabonne/llm-course

https://mlabonne.github.io/blog/

CSRF Token discussion with ChatGPT

https://chat.openai.com/share/796b43f6-20b6-4b22-b556-7c33d66db7ba

CSS - Using calc in a media query

 

Update to "The Rails and Hotwire Codex" book

https://binarysolo.chapter24.blog/the-update-to-the-rails-and-hotwire-codex/

ChatGPT discussion about ULID

https://chat.openai.com/share/e50b65c4-83da-4705-b122-70bb9894ab61

Installing Jupyter Notebook

https://python.land/data-science/jupyter-notebook

c2pa.org - Coalition for Content Provenance and Authenticity

https://c2pa.org/

The Coalition for Content Provenance and Authenticity (C2PA) addresses the prevalence of misleading information online through the development of technical standards for certifying the source and history (or provenance) of media content. C2PA is a Joint Development Foundation project, formed through an alliance between Adobe, Arm, Intel, Microsoft and Truepic.

Good website for learning frontend javascript

https://dmitripavlutin.com/

Rails: class_attribute

https://chat.openai.com/share/c06fff63-86d4-43c6-b257-196118054439

Rails - ActiveRecord - inverse_of

https://chat.openai.com/share/fc1f0d8b-8917-4f0c-8f30-036b0955f04c

Kafka with AVRO vs., Kafka with Protobuf vs., Kafka with JSON Schema

https://simon-aubury.medium.com/kafka-with-avro-vs-kafka-with-protobuf-vs-kafka-with-json-schema-667494cbb2af

Web API Design series - July 2022

https://www.youtube.com/playlist?list=PLP_rkG1reBjrCKy2Pb1bvjJKbKfantijk

Leetcode type problem - Sub-Arrays problem

 Generate number of sub arrays with a particular sum: E.g. nums = [1,1,1] with sum 2

arr = [1,1,1]
SUM =2
#arr = [1,2,2,3,4,5]
#SUM = 5

sub_arrays = []
arr = arr.sort

arr.each_with_index do |n, index|
  if n == SUM
    sub_arrays << [n]
    next
  end
  rest_of_array = arr[(index+1)..-1]
  temp_arr = [n]

  rest_of_array.each do |m|
    temp_arr << m
    if temp_arr.sum == SUM
      # We found a sub_array!
      sub_arrays << temp_arr
      # There might be duplicates of m, so let us remove m from temp_arr and continue
      temp_arr = temp_arr[0..temp_arr.length-2]
    end
    if temp_arr.sum < SUM
      # do nothing; we can continue to add more elements
    end
    if temp_arr.sum > SUM
      # discard all elements except n
      temp_arr = [n]
    end
  end
end

puts sub_arrays.to_s

Cryptography Foundations

 Below is an excerpt from the book - Securing Cloud Applications (manning.com)

The 15 biggest data breaches of the 21st century

https://www.csoonline.com/article/534628/the-biggest-data-breaches-of-the-21st-century.html

Optimizing HTML to PDF in Node.js

https://blog.logrocket.com/optimizing-html-pdf-node-js

The complete guide to Kubernetes cost management in 2023

https://www.spectrocloud.com/blog/the-complete-guide-to-kubernetes-cost-management

State of Cloud Security

https://www.datadoghq.com/state-of-cloud-security/

Amazon’s free courses on generative AI

https://www.aboutamazon.com/news/aws/aws-free-ai-skills-training-courses

The Architecture Of Serverless Data Systems

https://jack-vanlightly.com/blog/2023/11/14/the-architecture-of-serverless-data-systems

Load Shedding for High Traffic Systems

https://www.codereliant.io/load-shedding/

React Server Components: A comprehensive guide

https://blog.logrocket.com/react-server-components-comprehensive-guide/

Developing an effective CI/CD pipeline for frontend apps

https://blog.logrocket.com/best-practices-ci-cd-pipeline-frontend

Create Custom GPTs

 

OpenAI & Microsoft's HUGE AI Announcements (Dev Day Supercut) - Nov 2023

 

Software Design by Example

https://third-bit.com/sdxpy/

Rails and JS - No Build for JS

https://world.hey.com/dhh/you-can-t-get-faster-than-no-build-7a44131c

We're making it using vanilla ES6 with import maps for Hotwire, and vanilla CSS with nesting and variables for styling. All running on a delightfully new simple asset pipeline called Propshaft. 

Rails - Spree - Promotions - show a popup with some products when a certain condition is met

https://chat.openai.com/c/e1c29d26-f19e-472d-b701-262c8fca1645

Strada released

https://world.hey.com/dhh/finishing-hotwire-with-the-introduction-of-strada-9efdf091

https://strada.hotwired.dev/

Rails: class_attribute

https://chat.openai.com/c/388e1263-46a6-4406-b16d-8094b8232dfb

In essence, whenever you find yourself thinking, "I need a class-level configuration that should have a default, but I also want the flexibility to customize it for certain subclasses or instances," class_attribute is a tool you might consider using.

Rails - ActiveRecord - inverse_of

https://chat.openai.com/share/e8f26c7d-d539-4ba9-93b3-52f702ca8909

Rails: Join multiple nested associations

Join multiple nested associations:

https://gist.github.com/abhionlyone/84dec0aa7a5d30b9e2bc6d7bd2094f20

Rails 7.1 - Authentication from Scratch

https://gorails.com/episodes/rails-7-1-authentication-from-scratch

KEDA - autoscale your Kubernetes workloads in an event driven way

https://www.padok.fr/en/blog/keda

MySQL: Importing & Exporting a database via mysql cli

https://chat.openai.com/share/31c2512f-8545-4e90-888b-91ee8a40a22b

'Nested set model' data structure - ChatGPT discussion

https://chat.openai.com/c/14a54f06-d8e7-42af-b60e-402d821412be

Adjacency List Model vs Nested Set Model -  https://www.youtube.com/watch?v=0G58ot8obFs

React Native project - Sep 2023 - Deliveroo Food Ordering Clone with React Native (Expo Router, Zustand, Reanimated, Google API)

 

Ruby 3.0 - super() and yield() have been removed

https://g.co/bard/share/6bdde26fd5a6

https://g.co/bard/share/bb7128a02c84

Tech stack at Zerodha

https://zerodha.tech/

AWS - Progression from EC2 to EKS

https://chat.openai.com/share/b849cb14-0860-4d91-b802-fec54a6ea8af

OAuth 2.0 Flows

https://twitter.com/alexxubyte/status/1696180531266715815?s=20

Redis project

https://rohitpaulk.com/articles/redis-3

Solution to Redis project (step 4)

codecrafters.io - Practice Writing Complex Software

codecrafters.io

What are Passkeys?

https://www.tomsguide.com/news/what-are-passkeys

https://tailscale.com/blog/passkeys/

Warpstream Blog - Kafka is dead, long live Kafka

https://www.warpstream.com/blog/kafka-is-dead-long-live-kafka

What is Machine Learning?

https://twitter.com/alepiad/status/1684264492841193482?s=61&t=Q-HuKADa-FRvv_WSxlBOLA

https://techwriters.info/apiad/foundations-of-ml-1

Rails Interview Book

https://nezirzahirovic.gumroad.com/l/bible?layout=profile

Book - Master Ruby Web APIs

https://devblast.com/r/master-ruby-web-apis/toc

Algorithms Illuminated Omnibus Edition (September 2022)

https://www.algorithmsilluminated.org/

Speedrun a Realtime Chat App with Ruby on Rails 7

 

Using ChatGPT with YOUR OWN Data. This is magical. (LangChain OpenAI API)

 

Rails design patterns - The big picture

https://rubyhero.dev/rails-design-patterns-the-big-picture

What’s new for SwiftUI in iOS 17?

https://www.hackingwithswift.com/articles/260/whats-new-in-swiftui-for-ios-17

Common Table Expressions in Active Record

https://www.hashnotadam.com/posts/2023/05/common-table-expressions-in-active-record/

Kubernetes Security Basics

https://medium.com/@AlexanderObregon/kubernetes-security-basics-efee6f670e9f

Deep Dive into etcd: A Distributed Key-Value Store

https://medium.com/@extio/deep-dive-into-etcd-a-distributed-key-value-store-a6a7699d3abc

MIGRATION FROM ON PREM TO THE CLOUD IN AWS: Concepts to understand and Steps to follow.

https://medium.com/@augustineozor/migration-from-on-prem-to-the-cloud-in-aws-concepts-to-understand-and-steps-to-follow-131e79c0ecf5

Is React Having An Angular.js Moment?

https://marmelab.com/blog/2023/06/05/react-angularjs-moment.html

Amazon Prime Video switches to Monolith from Microservices

https://www.primevideotech.com/video-streaming/scaling-up-the-prime-video-audio-video-monitoring-service-and-reducing-costs-by-90

React Server Components discussion by Dan Abrimov - Apr 2023

https://twitter.com/dan_abramov/status/1651699851120242689?s=61&t=JTieX__4Y1hUB18J8I48cw

The Modular Monolith: Rails Architecture

https://medium.com/@dan_manges/the-modular-monolith-rails-architecture-fb1023826fc4