/tech/Recent content in Tech on Devops DiaryHugo -- gohugo.ioen-usTue, 09 Feb 2021 00:00:00 +0000/tech/s3-elasticsearch-snapshots/Tue, 09 Feb 2021 00:00:00 +0000/tech/s3-elasticsearch-snapshots/We use elasticsearch for a lot of purposes within our organisation, and thanks to kibana, we sometimes also use elasticsearch to store some data that isn’t necessarily timeseries data, but just nosql data that we want to visualize. We had a business requirement of having multiple backups for one of these types of data. So, we decided to have a dump of these indices once everyday and store it in an Amazon S3 bucket, in case the worst should happen./tech/logging-at-scale-2/Fri, 12 Jun 2020 00:00:00 +0000/tech/logging-at-scale-2/Continuing from the last post where we created a logging pipeline for storing and visualizing access logs in realtime, we faced some further issues with the said pipeline. In this post, I’ll try to build on that and get rid of some of these issues. An expected unexpected problem With the increase in number of requests during spikes in traffic, we were running quite a few instances. And that’s when we first encountered the AWS API request throttling./tech/logging-at-scale/Fri, 08 May 2020 00:00:00 +0000/tech/logging-at-scale/So I work for an Ad-Tech company, and in this industry we receive a huge volume and rate of requests. At our peak, we serve around 35,000 requests per second, which is actually a bigger number than you initially realize. Over the past 2 years, our business has seen an exponential growth in the amount of requests we serve and the infrastructure we handle. When I joined 2 years back, we were serving like 400-500 qps and handled like 20 servers for different purposes./tech/setup-own-dhcp/Sun, 03 May 2020 00:00:00 +0000/tech/setup-own-dhcp/On a network with a large number of clients, DHCP is a must for dynamically assigning IP addresses to all the connected clients in order to avoid address conflicts and automating other required configurations like subnet mask, default gateway, DNS servers etc. Nowadays, almost all of the routers, even the smaller home routers already have an inbuilt DHCP server within the router that assigns IP addresses to connected clients. Still, why not learn how to set up one for yourself?/tech/blog-setup-part-2/Thu, 16 Apr 2020 00:00:00 +0000/tech/blog-setup-part-2/So, in the last post, we finished setting up the infrastructure for our blog. In this post, we’ll deploy our blog to our server and make our blog live! The idea here would be to do all this within the User Data Script, which is executed when the EC2 instance is initializing. We’ll write a shell script with all the required commands and run it with the User data script. Here are the things that need to be done to make our blog live:/tech/blog-setup-part-1/Wed, 08 Apr 2020 00:00:00 +0000/tech/blog-setup-part-1/So, after a long time and a very little deliberation, I finally decided to stop using Google Blogger and move to a “self-hosted” soultion. The idea was to setup most of the things from scratch and have much more control over my own blog. So, I decided to give “static site generators” a try. I looked at a few different static site generators and in the end decided to go with Hugo, because it was version controlled, fast, secure, easy to understand and had good flexibilty in terms its functionality (not to mention a lot of themes to go with it)./tech/setup-own-vpn/Thu, 06 Feb 2020 00:00:00 +0000/tech/setup-own-vpn/VPN services are pretty popular these days. As days pass, more and more people are starting to use VPN services. There are a lot of benefits for using VPNs, some of them being: Protection from snooping on public networks (like public WiFi hostspots etc.) Bypass geographic restrictions on websites. Access blocked websites and services. Hide your internet activity from your ISP. Avoid network throttling (in some cases) and so much more…./tech/cyberoam-autologin/Sun, 18 Jun 2017 00:00:00 +0000/tech/cyberoam-autologin/Most of the universities and institutions nowadays use cyberoam to control and monitor the way their students/employees use their Internet connection. While using VPNs can help in overcoming the whole login process altogether, but many times it doesn’t work. Now, being an CS student, I can’t really live without Internet for long and it’s pretty annoying when cyberoam logs out in between a streaming football match, or while downloading a really large file./tech/downloading-twitter-images/Sun, 10 Apr 2016 00:00:00 +0000/tech/downloading-twitter-images/So, continuing my exploration of the twitter API using tweepy, I realized exploring different user’s tweets is ridiculously easy. So, what exciting things can one do with these tweets?? So here I was, scrolling twitter in my free time when I came across a lovely Özil wallpaper. Generally Arsenal’s twitter feed includes some great pictures. Similarly for movie stars or artists, who post their pictures on twitter. It takes a lot of time to download them manually./tech/windows-10-install/Tue, 30 Jun 2015 00:00:00 +0000/tech/windows-10-install/Its been a little over a week since Microsoft released Windows 10 for the general public, and more than 14 million people worldwide have upgraded their PCs to Windows 10. With Microsoft announcing that the upgrade from Windows 7, 8, 8.1 would be free for a year, everyone is upgrading to the new and improved version of Microsoft’s Windows. Whether it is the customizable start menu, the sleek new looks or the newly introduced Microsoft’s voice assistant Cortana, windows 10 is high in demands.