Gaurav Sen
Gaurav Sen
  • Видео 295
  • Просмотров 28 852 294
30 [Software Engineering] research papers you should read
You can find all the papers listed here:
interviewready.io/blog/white-papers-worth-reading-for-software-engineers
Here are 30 whitepapers worth reading for software engineers. These papers are dense sources of information.
Most are from Google and Meta, but you see the occasional Amazon or LinkedIn paper. Keep an eye out for the topics you like.
Gotta read them all!
00:00 Whitepapers
00:12 30. Scalability at what COST
00:50 29. Detection of Silent Data Corruption
01:20 28. Prophet
01:57 27. Napa
02:26 26. Cubrick
03:00 25. Monitoring and Root Cause Analysis
04:00 24. Presto
04:40 23. Foundation DB
05:16 22. F1 Lightning
06:16 21. LinkedIn Distributed Graph
07:15 20. TensorFlow
07:30 19. The monolith strik...
Просмотров: 4 672

Видео

Find the distance between friends/connections - LinkedIn System Design with @KeertiPurswani
Просмотров 7 тыс.Месяц назад
Have you noticed a small icon on the right of LinkedIn profiles? It tells you how closely you are related to a user (1st, 2nd or 3rd-degree connection). These distances are calculated using graph algorithms. Specifically, LinkedIn uses Bi-directional BFS. This is a two-way algorithm where a search is started from both source and destination, and terminated midway. The total distance traveled fr...
System Design of a Delivery System like Zomato with @KeertiPurswani
Просмотров 12 тыс.Месяц назад
In this video, we look at what powers the delivery and tracking systems of e-commerce apps like Zomato, Swiggy, and Amazon. Along the way, we go through relevant spatial algorithms, network protocols, and design components used in location-based systems. Cheers! 00:00 Problem Statement 00:45 Requirements 02:00 High-Level Design 04:01 Capacity Estimation 05:36 Location-Based Queries 13:30 Tracki...
System Design of GitHub Code Search - SDC Episode 1 with @KeertiPurswani
Просмотров 10 тыс.2 месяца назад
Github code search allows developers to view and edit code online. This is particularly useful when debugging code in remote locations (like on vacation)! GitHub manages permissions, storage, and retrieval through a set of services. In this video, we look at systems that power its search APIs. If you have any doubts or suggestions, please share them in the comments below. This is the first epis...
Latency Numbers Every Programmer Should Know - 1000x slow-down
Просмотров 11 тыс.5 месяцев назад
These are some latency numbers every programmer should know. 00:00 Memory and SSD 00:13 Sequential Disk Read 00:33 Roundtrip CA to Netherlands 02:48 Roundtrip CA to India 06:08 Ends Git Gist: gist.github.com/jboner/2841832 System Design: interviewready.io/ Designing Data-Intensive Applications Book: amzn.to/3SyNAOy Twitter: gkcs_ #SystemDesign #Software #Coding
The Most Advanced Multiplication Algorithms: Why Karatsuba and FFT beat high-school mathematics
Просмотров 4,6 тыс.6 месяцев назад
For folks interested in multiplication algorithms (turns out this is a big deal in computer science and math). 1. High school method - O(n^2) 2. Karatsuba - O(n^1.58) 3. Toom-Cook - O(n^1.46) 4. FFT - O(n*log(n)*log(log(n))) The last algorithm is pretty complex. We explain the intuition behind it in this video. 00:00 Who should watch this? 00:18 Example Problem 01:23 Karatsuba Algorithm 02:38 T...
Coding Interviews must be FIXED [Open to Thoughts and Suggestions]
Просмотров 14 тыс.6 месяцев назад
Software Interviews need a revamp. Coding rounds are IRRELEVANT to our jobs. On average, software developers spend three months preparing for an interview. A good engineer has to choose between: 1. Focusing on the job. 2. Practice useless DSA questions. It's time to ask job-relevant questions and test candidates' WORKPLACE capabilities. At InterviewReady, we are building tools to test engineeri...
How ANOMALY DETECTION works in time series using the Holt-Winters Algorithm
Просмотров 4,2 тыс.6 месяцев назад
This video explains how anomalies are detected in a time-series graph. The algorithm's name is Holt-Winters. The idea is simple, and the results are often useful. Cheers! 00:00 What is a time series? 00:30 How can we find anomalies? 01:00 Anomaly Detection Algorithm 02:25 Repeated Differentiation 03:36 Example 04:14 Thank you! #anomalydetection #timeseries #monitoring InterviewReady: interviewr...
20 AWS services you should know [as a Software Engineer]
Просмотров 27 тыс.7 месяцев назад
Here are some AWS services you should know as a software developer or freelancer. These services can help you quickly build and manage the technical muscle of your organization! InterviewReady: interviewready.io/ Each of these services solves a different business and tech problem. Let's dive in! 00:00 Who should watch this? 00:32 20. AWS Neptune Graph Database 01:17 19. AWS Cognito Authenticato...
A SUPER FAST tech stack for startups [using AWS and Payment Gateways]
Просмотров 20 тыс.7 месяцев назад
A SUPER FAST tech stack for startups [using AWS and Payment Gateways]
20 Whitepapers that changed the world [For Senior Software Engineers]
Просмотров 127 тыс.7 месяцев назад
20 Whitepapers that changed the world [For Senior Software Engineers]
Design Patterns for High Availability: What gets you 99.999% uptime?
Просмотров 21 тыс.7 месяцев назад
Design Patterns for High Availability: What gets you 99.999% uptime?
Tech that DOESN'T WORK for Start-ups and Mid-sized companies
Просмотров 17 тыс.8 месяцев назад
Tech that DOESN'T WORK for Start-ups and Mid-sized companies
Authorization across Distributed Systems: The OAuth Protocol
Просмотров 19 тыс.8 месяцев назад
Authorization across Distributed Systems: The OAuth Protocol
How Stored Procedures make databases BLAZINGLY FAST
Просмотров 16 тыс.8 месяцев назад
How Stored Procedures make databases BLAZINGLY FAST
Garbage Collection Algorithms in Java: Concurrent Updates with Optimistic Locking - Part III
Просмотров 8 тыс.8 месяцев назад
Garbage Collection Algorithms in Java: Concurrent Updates with Optimistic Locking - Part III
How 100 milliseconds cost Amazon 3 BILLION DOLLARS: Latency, Concurrency and Parallelism
Просмотров 50 тыс.9 месяцев назад
How 100 milliseconds cost Amazon 3 BILLION DOLLARS: Latency, Concurrency and Parallelism
Impeccable API Design: What you MUST CONSIDER before deploying APIs to production
Просмотров 22 тыс.9 месяцев назад
Impeccable API Design: What you MUST CONSIDER before deploying APIs to production
What is an API Gateway?
Просмотров 46 тыс.9 месяцев назад
What is an API Gateway?
Career Choices: Four differences between Managers and ICs
Просмотров 13 тыс.11 месяцев назад
Career Choices: Four differences between Managers and ICs
How custom banners are shown INSTANTANEOUSLY: Contextual Advertising at Scale
Просмотров 22 тыс.11 месяцев назад
How custom banners are shown INSTANTANEOUSLY: Contextual Advertising at Scale
Running a bootstrapped startup for 3 Years: The story of InterviewReady!
Просмотров 8 тыс.11 месяцев назад
Running a bootstrapped startup for 3 Years: The story of InterviewReady!
Dark Mode: Pilot Launch at InterviewReady
Просмотров 2 тыс.11 месяцев назад
Dark Mode: Pilot Launch at InterviewReady
Zerodha Stock Broker System Design with @KeertiPurswani
Просмотров 30 тыс.Год назад
Zerodha Stock Broker System Design with @KeertiPurswani
Task Management - The one thing you must know as a software engineer
Просмотров 16 тыс.Год назад
Task Management - The one thing you must know as a software engineer
10 steps in the career of a software engineer: From SDE-1 to Principal Engineer
Просмотров 84 тыс.Год назад
10 steps in the career of a software engineer: From SDE-1 to Principal Engineer
Front-End to Full-Stack: 9 Different Types of Software Engineers
Просмотров 24 тыс.Год назад
Front-End to Full-Stack: 9 Different Types of Software Engineers
8 Rules for Offer Negotiations by Gaurav Sen
Просмотров 62 тыс.Год назад
8 Rules for Offer Negotiations by Gaurav Sen
Let's get InterviewReady!
Просмотров 19 тыс.Год назад
Let's get InterviewReady!
CS Grad Placement Roadmap: First Year to Fourth Year
Просмотров 16 тыс.Год назад
CS Grad Placement Roadmap: First Year to Fourth Year

Комментарии

  • @kiyotaka9135
    @kiyotaka9135 2 часа назад

    is the formula right i think it is wrong i am talking about the C value should'nt it be outside the root

  • @cocorita
    @cocorita 8 часов назад

    ah... nothing like blaming co-workers for errors to be a proper Indian developer LOL

  • @darkstudio3170
    @darkstudio3170 10 часов назад

    I am not really sure how girls get into big tech despite having this bad at designing

  • @pepehimovic3135
    @pepehimovic3135 10 часов назад

    Around 8:22, should it not be the height of each sub tree, not the size? I reason that that should be more efficient since we want to maximize the size of each chain

    • @pepehimovic3135
      @pepehimovic3135 10 часов назад

      8:46, yes, we want to attach as many nodes to any given segtree (greedily), so why use size of sub trees and not height?

  • @user-qn7lt2xb1n
    @user-qn7lt2xb1n 10 часов назад

    Web sockets using XMPP. Right?

  • @demogyani2419
    @demogyani2419 11 часов назад

    These kind og System design interviews are more about showoff. Just learn about load balancer, autoscalling, microservice, kafka, logging service, databse replica draw some stupid diagrams and u can crack it. I personally feel like it should be more about database design, choosing the right set of technology, api security, architecture of ur code. Choosing between different databse system.

  • @user-qn7lt2xb1n
    @user-qn7lt2xb1n День назад

    Can we store messages in some queue if the user's connection id is not active (user is offline) When use comes online, the message is published to him and popped out of the queue simultaneously instead of storing it in the db?

  • @premkumar-wt8lu
    @premkumar-wt8lu День назад

    @Gauvravsen Thanks for the wonderful video. Can you please ebay auction system design

  • @navinrangar2626
    @navinrangar2626 День назад

    Unfollowing you. Your t-shirt says.

  • @hyperme1831
    @hyperme1831 2 дня назад

    Anybody had done with system design interview at any company of 2 to 3 years experience? Please tell whag is expected in interview we need to write classes and main funftionaloty code as well or if we explain everything by diagram thats fine

  • @tsarprince
    @tsarprince 2 дня назад

    Isn't it that last seen should be updated by an onClose app listener instead of being dependent on the last user request to gateway? Would be lesser requests and more accurate imo.

  • @shubhamkalla6489
    @shubhamkalla6489 2 дня назад

    which tool is used for online drawing can someone help

  • @Vlad-xo8hm
    @Vlad-xo8hm 2 дня назад

    Thank you for the video sir!

  • @Bob_tygabk
    @Bob_tygabk 2 дня назад

    Hey do u do one one one prep for system design interview (particularly for stripe) if not stripe then similar company like PayPal How can I set up something like that ? I have a system design interview soon and I will like to prep with u

  • @tp1720
    @tp1720 2 дня назад

    Great Job Gaurav. Thanks so much !!

  • @ShubhamKalla-jq4en
    @ShubhamKalla-jq4en 4 дня назад

    Which tool has been used for drawing the archtiecture

  • @user-mx2bk2fq4s
    @user-mx2bk2fq4s 4 дня назад

    make a video on you favourite websites for blogs, articles, papers etc.

    • @gkcs
      @gkcs 4 дня назад

      github.com/InterviewReady/system-design-resources

  • @deepak655655
    @deepak655655 4 дня назад

    Gaurav talking about UDP and his voice went berserk, talking about coincidence !!!

  • @jhonsen9842
    @jhonsen9842 4 дня назад

    you should have added Mapreduce that changed the Computing and evolved big data.

    • @gkcs
      @gkcs 4 дня назад

      Yeah that's a great one too.

  • @debjitbhattacharjee5926
    @debjitbhattacharjee5926 4 дня назад

    We missed dynamo from amazon?

  • @RishiRajxtrim
    @RishiRajxtrim 4 дня назад

    👍

  • @NeverMyRealName
    @NeverMyRealName 4 дня назад

    where do you find these papers? also links would be helpful

    • @gkcs
      @gkcs 4 дня назад

      They are in the blog mentioned in the description. The links were exceeding the RUclips description limit.

  • @loupa6128
    @loupa6128 4 дня назад

    I haven't read much papers, but one I found very interesting and insightful was "R-trees: a dynamic index structure for spatial searching" from Antonin Guttman.

    • @gkcs
      @gkcs 4 дня назад

      I'll look into it, thanks!

  • @101abhay
    @101abhay 5 дней назад

    I would be great if you shared research papers links

    • @gkcs
      @gkcs 4 дня назад

      It's the top link in the description 🔗

  • @Parijatnagar
    @Parijatnagar 5 дней назад

    Its new for me.

  • @Parijatnagar
    @Parijatnagar 5 дней назад

    😊❤❤

  • @killeraloo3247
    @killeraloo3247 6 дней назад

    I think we can use a combination of monolithic and microservices architecture.

  • @rashmeemayeemohapatra5604
    @rashmeemayeemohapatra5604 6 дней назад

    Appreciate you for your efforts!

    • @gkcs
      @gkcs 5 дней назад

      Thank you!

  • @i.sapnadip
    @i.sapnadip 7 дней назад

    So does this mean ,Backup servers? like h1(0) h2(0)????

  • @shreyasgtrivikram8467
    @shreyasgtrivikram8467 7 дней назад

    Topic is Explained very nicely . Would it be possible for you to provide the link for the free resource on CDN's on how they deliver live streaming . Would be really helpfull Thank you 😇

  • @anuradhadas8795
    @anuradhadas8795 7 дней назад

    "Indexing is cute" is my new go to phrase.

    • @gkcs
      @gkcs 5 дней назад

      Database speeds up by 300%.

  • @vikashsingh8312
    @vikashsingh8312 7 дней назад

    Bro did not say Kafka at any cost Lol

  • @mitgundigara466
    @mitgundigara466 7 дней назад

    I personally felt this is too abstract however it's good place to start. There are multiple scenarios which is not handled and it's assumed that it's happy case. I request to go slight deeper in each feature. The design explained in the video is "common sense", any software developer would have think of it. But I really request if you can answer some of the below questions which interviewer can ask us during the interview, if we would have explain the same. 1) You first explained we would store the mapping of each client to the gateway box. But there will be multiple clients will be connected to same gateway. let's say A is on gateway no 1 and B & C is on gateway no 2. then let's say A sends message to B. How would gateway-2 know where we need to route the message for B and C. My hunch is gateway will be storing this information. 2) I really wanted to understand when you will use Gateway vs Load Balancer. In your all videos somewhere you have used gateway and some places you used Load Balancer. You should explain why gateway over load balancer. 3) How are you maintaining the sequence of the message is not explained, this should be a very basic message. A and B can see different view of the message. 4) Why do we need message queue to store the failed message? IMO we should maintain 3 DBs. Sent, Delivered & NotDelivered with userId as key and Map<UserId/GroupId, List<Message>>. This will be easier. Whenever A sends message and chat server receives the message you can probably initiate two thread where one thread store message in NotDelivered table and the other thread calls SessionService to fetch the B's gateway box and send the message. Once B send acknowledgement then we can remove message from the NotDelivered and put in Delivered table. You can delete the message from the DB and archive the message in the low cost storage like S3 after 30 days or data is backup. Generally it happens every day night. The advantage of this is let's say B is offline then we do have store all the message that B got from all user's in B's contact list in Not Delivered table. Once B turns internet on their app will send long polling request and then can easily fetch all the data. Is this good design? I might be wrong but this is my thought. 5) You should mention what data we can cache and at which layer. 6) I think you should be more clear when your connection will be active, when it will not. I think if someone has opened whatsapp application or as long as it is there in app background stack the connection should be open. Once we remove the application from the background stack then connection will be lost. Once the connection is lost then your application will continue to send the long polling request until internet is on at certain interval. This kind of arch will have benefit of both websocket and long polling. Once internet is off it can't do even long polling. 7) What type of database and what are the data we are storing. I read in grokking system book that we can use Cassandra (wide column family) But I didn't understood why. So What I am suggesting if possible rather than making too much abstract information, make videos little more systematic it can help us. I think there are multiple counter questions interviewer can ask. why to choose this database over other. what data you would cache, which cache policy would you use (write back/around/through). 8) Here API gateway is just routing the chat server right? It's very difficult to put all the information together. But making video more systematic would certainly help me and others. You can have define structure while making system design video. at least following structure might help better. If you can even explain 5 mins for each of the section could be very helpful to all of us doing system design preparation. 1) Requirement 2) Define constraint/limitation/ data constraint like no of server, Incoming/outgoing data/memory storage 3) High level design for each feature 4) API Gateway/ Load Balancer 5) Data layer what data you will be storing 6) Caching 7) Sharding

  • @narissakarimahajratalli6779
    @narissakarimahajratalli6779 7 дней назад

    can you go more into the debate btw using a no SQL database vs horizontal partioning?

  • @adityaagrawal3576
    @adityaagrawal3576 8 дней назад

    This is an awesome video. I am building start up and this is like a elder brother giving advice

    • @gkcs
      @gkcs 7 дней назад

      Best of luck!

  • @shubhamuniyal9047
    @shubhamuniyal9047 8 дней назад

    Currently I'm just an ex😢

  • @rahulkushwaha7896
    @rahulkushwaha7896 8 дней назад

    why not always devide the server equally on the ring, on any event for addition and removal. ?

  • @kuldeepsharma4543
    @kuldeepsharma4543 9 дней назад

    why didn't she use Load Balancer?

  • @saypal18
    @saypal18 10 дней назад

    I am thinking of a different method: we have a single hash function h(r) and if there are k servers, we compute all the hash values h(r)%1, h(r)%2, ... , h(r)%k, and store it in an array. Now loop through the array in reverse and the first time we get array[i] = i-1, we send it to the ith server, and of course come out / break the loop Let's prove this via induction. Its trivial for only 1 server. Let's say that we have n servers, and load was equally distributed among them. Let's introduce a new server n+1. The new value added to the array for any given request is h(r) % (n+1). in only 1/n+1 evenly distributed cases, will its output be n, and request will be directed to the n+1th server.. Since we assumed that previous load was evenly distributed, load to the new server will also be taken evenly from all previously existing servers.

  • @NallaSanjayReddy
    @NallaSanjayReddy 11 дней назад

    Nice interactive session

  • @gtasa6000
    @gtasa6000 12 дней назад

    While I do think stored procedures are good (still evaluating), I think this video exhibits the advantage of trying to use single query (employing joins or any other way) over executing multiple queries over the network.