Breaking into Product Management – a guide

Over the last 2 weeks, 7 friends have asked me how they can break into Product Management role. These are software developers, designers, fresh graduates and young entrepreneurs, and my advice to all of them has been more or less the same. Additionally, I have got an opportunity to interview several people from product management roles over the last year where I see first time candidates making the same mistakes over and over again.

Hence, I have decided to jot down a guide on landing your first PM job. Its made up of 3 parts, answering three important questions –

  1. Is Product Management right for you?
  2. How to land a Product Manager job?
  3. How to spend your first 90 days as a Product Manager?

This is a summary of my key learnings and aims to give you a framework to think about making the switch. These are not general and not facts. I have tried to not reinvent the wheel and give links to the right resources wherever I could. Hope it helps!


Part 1 : Is Product Management right for you?

This is the first step. I remember searching across the internet trying to understand what a PM does and landing onto this universal representation of a PM role –

What, exactly, is a Product Manager? - Mind the Product
The universal PM Venn diagram

Problem with this is it’s too superficial and does not tell much about what qualities, characteristics, traits or interest will really help you excel as a PM. In this post, we try to explore that in detail.

Role & Success criteria of a PM

A product manager’s role is to define a product (or a product line, in some cases) and do whatever it takes to make it a success. Success is broadly defined based on the business impact created by the product which could be measured in different ways based on the type of business.

B2B vs. B2C

At a very high level, a business either sells to consumers (B2C) or enterprises (B2B). The key difference is that in case of B2B – your buyers (decision-makers) are not actually the users of your product. So, subscription, retention and reliability matters more. Whereas in B2C, your buyer is your end user (may not be if e.g., you are selling dog biscuits, but the buyer is still in very close contact with the end user), hence scale is huge and virality, user retention and UX are the prime focus.

Metrics by Business Models

Lean Analytics has pretty detailed description of various business models and the key metrics that matter in each. Summarising briefly below –

Business ModelMetrics that Matter
E-CommerceConversion rate, purchases/year, avg cart value, abandonment, Customer Acquisition Cost (CAC), Customer Lifetime Value (CLV)
SaaSARR, CAC, Churn, New Registrations, Conversions, Stickiness, CLV
Mobile appDownloads, CAC, % active users, virality, churn, CLV, avg users per month, time to first purchase
Media/Content SiteAudience size, Content availability, Churn, Ad inventory, Ad rates, CTR, Stickiness
User generated content (community)Content creation, Content value, % of engaged users, virality, engagement funnel shifts
Two-sided marketplaceBuyer/Seller growth, Inventory growth, Search Effectiveness, Conversion funnel, Ratings and signs of fraud, Pricing metrics
Lean Analytics – Business models and Metrics that matter

So, as a PM, your success will be measured by how your product(s) impact the above metrics based on the kind of business that you are in, and your role will be to drive all stakeholders across marketing, sales, engineering, customers, users, etc. to –

  1. Identify the most critical product problems to solve to achieve business goals
  2. Decide what to build & how it should work
  3. Make sure it gets built as planned and on time
  4. Drive communication & demand generation with growth teams
  5. Drive adoption and collect data on usage
  6. Use the data to evaluate product success/failure & scope for improvement
  7. Plan the next steps, align your leadership and go back to step 1

Since you have seen enough Venn Diagrams on PM already, here is a breakdown of Automattic’s design process which I think is quite relevant to product development process –

Four planets of product development by Automattic

What skills help a PM succeed?

Looking at the above Venn diagram, a lot of beginners feel that PMs need to be great at design, business and tech before they even start their journey. However, that is usually not the case. A lot of PMs have T-shaped skillsets which means they are great at one thing and good enough in several others. I personally think that it’s important to be deep enough into one of these three to be able to make a mark in the interviews. See 10-30-50 rule [1].

In this section, we will break down PM skills into soft and hard skills and list down some of the top skills that make good PMs.

Hard skills of a good PM

  1. Intellectual ability to process & synthesize a lot of information
  2. Systems thinking & analytical skills
  3. Comfortable with quants and data
  4. Technical background
  5. Comfortable with UX/Design

Soft skills of a good PM

  1. Influential communication
  2. Demonstrated leadership (execution, alignment)
  3. Extreme Ownership
  4. Product Sense [2]
  5. Truth-seeker mindset (hypothesis creation & testing)
  6. Perseverance to always show up
  7. High Agency (ability to solve any blocker to get stuff done – all day, every day) [3]

Shreyas Doshi has created this comprehensive mindmap listing all possible PM skills –

Image

I learned a lot about this from Todd Jackson’s post. Would highly recommend [4].

Common questions (and myths) about being a PM

Do I need to know to code to be a PM?

You don’t need to know how to write code. However, if you are working on a tech product, knowledge of the basic principles of computer science is essential, so that you can communicate effectively with developers. Having said that, you should definitely aim to learn the basics of programming and computer science over time as it’ll help you get better at solving issues and removing tech blockers. It also depends on how technical the product is, e.g., a PM at a cloud storage company needs to be much stronger at tech than a PM at a mobile gaming app company. Cole Mercer’s course on Technology for PMs[5] is a good start.

Do I need to learn how to make wireframes to be a PM?

You don’t need to create beautiful designs, however wireframing is an essential skill that helps PMs communicate their ideas visually. So, would highly recommend to get your hands dirty on Sketch, Figma or whatever your tool of choice is. More than design, it’s important to develop user empathy and a great UX sense. Try solving Artiom Dashinsky’s design exercises [6]. More resources in Part 2, where we talk about how to prepare for the interviews.

Do I need to do an MBA to get into Product Management?

No, and if you’re looking for a PM role in a startup, absolutely not! If your long-term goal is to continue down the PM/founder (builder) path and not go into consulting, investment banking and other traditional MBA pursuits, I would highly recommend to save those 2 years and spend them learning about building better products by grinding with a good product team.

Is the PM the CEO of the product?

I literally cringe when I hear someone say that. IMO, PM is not the CEO of the product, because while PM gets to decide the product strategy, they do not have any say in how the engineering or growth resources will be used and their success solely depends on being able to convince the leadership and other teammates to go that route and take specific tradeoffs. So, when someone says this to you, it usually means, “we would like you to take as much ownership of this product’s success or failures as a CEO would.”

How to build credibility in my resume?

By building stuff or having strong opinions about how something can be made better. We will talk more about this in part 2.

What is Product Management not?

This deserves its own thread because a few companies rebrand multiple roles as PM to attract quality talent. To keep it short. –

  • PM is not a Program Manager.
  • PM is not a full-time developer/designer.
  • PM is not a Product Marketing Manager.
  • PM is not an engineering manager.
  • PM is not a business analyst for requirement gathering.

Career progression in Product Management

This is what a typical PM career path might look like. Each PM starts off as an individual contributor. However, just to emphasize, PM is by definition a fairly senior role (even if you are an IC). Based on how the company os structured, you may or may not be given P&L responsibility as part of the role.

However, since you learn a variety of skills as a PM, you actually don’t need to follow this path. A lot of PMs end up becoming CEOs (Susan Wojcicki, Youtube) or founders (Ivan Zhao, Notion) themselves.

The tough parts about being a PM

Like every other role, PM has its challenging bits as well and while there are ways to deal with them, its extremely important to know what you’re getting into before making the choice. Here I mention a few things I have experienced and validated from other PMs as well :
Disclaimer : These have a lot to do with the company that you work with. as well. Have seen it happen a lot lesser in companies with mature product orgs.

  1. Most of the people that you work with do not report directly to you – Responsibility? Yes. Authority? Umm..not so much.
  2. Have you ever heard – “PM ko aata kya hai” (what does a PM know)? They don’t build stuff, neither do they design or sell it? What the hell are they doing around here, giving gyaan on what we should build! To remind myself on how to add true value, I read this great post by Ken Norton atleast once a month – “Remember friend, nobody asked you to show up”. Well worth a read.
  3. It does get psychologically draining at times to show up and motivate yourself and your team everyday to push harder and create better stuff. Mental health is super important for PMs.
    “PM-ing is the kind of sales job where the sales call never ends. Where every moment is show-time!”
  4. You own all the criticism (from sales, customer, devs, marketing, etc.) but don’t really get as much appreciation because you actually neither built, nor designed the product!

Here is the visual description of what the journey of building any venture/product is like :

The Messy Middle by Scott Belsky

Why did I become a PM?

If you’re reading this, you are probably looking for an answer to whether you should give up whatever you’re doing and switch to a PM career. It’s hard for me to provide a generic answer to this. However, I can share my story of why I decided to give up my high-paying software engineering job at Arcesium (DE Shaw & Co.) to take up the role of a PM :

  1. I liked coding but I hated the fact that I could only work on one or two things at a time [6]. I wanted to widen the scope and work on several features at once.
  2. I felt that I was too far away from where the real money was made. I knew I couldn’t hope to be a great software dev if I came back from work to read strategy [7] blogs and business books every single day.
  3. I am a very purpose-driven person and the goal of making an app work did not excite me enough.
  4. I have been in leadership roles right from college where I was the first person to set up a team of 25+ to manage a makerspace[8]. I knew I liked mission-driven roles where I got to play a larger part.
  5. I am a genuinely curious person and love to read and write. I was. told that these would be useful skills as a PM [9] as its a role where you constantly learn.

That’s how I figured that this was the role that excited me the most and here I am! I hope this post helps you get some more clarity as well in terms of what future might look like if you do opt for a PM role.

Further Reading

  1. The 10-30-50 rule of building skills as a PM
  2. How to improve your product sense
  3. High-agency PM – what and how
  4. Todd Jackson on hiring great PMs
  5. Technology for PMs course on Lynda.com
  6. Building our own DSL for a Hackathon
  7. Stratechery by Ben Thompson
  8. Innovation Garage, NIT Warangal
  9. Good vs. Great PMs by Shreyas Doshi

We will follow this post with Part 2 and 3 where we talk about how to land a PM role and what to do to make the most of your first 90 days, respectively. Stay tuned!


Introducing KODA by Yellow Messenger

It is said thatย Scientia potentia estย orย Knowledge is Power. For large enterprises with a rich history and deep expertise of several decades in their industry, knowledge truly is a competitive moat. This competitive advantage, however, if left unmanaged, can soon turn into an overwhelming daily struggle for thousands of employees. Global enterprises suffer with what is known asย Knowledge Debtย โ€“ mismanaged, segregated information spread across legacy systems, which is hard to discover, access, update and collaborate on.

Sounds familiar?

– Yours truly, Yellow Messenger Blog

I have had the privilege of working first hand with large enterprises spread across 80+ countries with 100,000+ employees. These are organisations that have knowledge that theyย  created, discarded, repurposed and shared over 30+ years.ย  At Yellow Messenger, we have developed a multi-model solution to tackle the challenge of accessing real-time insights for an enterprise across structured and unstructured data – lakhs of documents, spread across systems and in multiple languages – you throw it at the insights engine and it gives you the most relevant content basis your query.

This is the result of a year of hard work from the entire team. We use a combination of search embeddings and Machine Comprehension to achieve this. I have written a detailed blog on this on our company website.ย  Do give it a read and would love to brainstorm with you on how to solve your knowledge management problem.

Cheers!

The importance of play

This was the result of a 1-hour writing marathon with Naveen Indala. The idea was to publish a blog in under an hour. Did this in Third Wave in Whitefield. #PeakBengaluru

Also, Naveen has started My Success Partner to help people get better every day at work and life. Do check it out!


The traditional concept of hard work (for me)

  • Hard work is the only way to succeed
  • Every minute ‘wasted’ enjoying can be put to better use
  • The more you think about a problem, the more time you spend on it, better are your chances of finding a solution.
  • Competitive nature/ego to grill, push yourself

How I learnt the importance of play

  • A week at Yellow Messenger where I was given a tough problem to solve and I couldnt stop thinking about it. I didnt shut my mind for 10 straight days – thinking only and only about it (not necessarily always working on it).
  • Ruminating – only made the problem look bigger, didnt necessarily fast-tracked the solution and left me exhausted, in my own echo chamber of thoughts.

Play improves performance at work

  • Moving mountains requires marathons, not sprints
  • Consistency >>> Flash of inspiration/genius
  • Play gives an opportunity to gather multiple outsider perspectives (MoP)
  • Meaningful conversations give opportunities to reflect and improve
  • Difficult problems require huge amount of resilience, loneliness and finding order in chaos – which is mentally draining
  • Play improves the overall quality of life – gives you a chance to meet people, learn new things, widen your point of views, shorten your blind spots
  • Play gives memories, enhances creativity and helps us be more human at work

Challenges while incorporating play in modern life

  • The internal urge to jump at a challenging problem and solve it quickly (sprint vs marathon)
  • The tyranny of convenience leading to loneliness, endless scrolling and fewer experiences
  • Carrying emails and linkedin in our pockets ensures that work never leaves us mentally
  • Not all kinds of play is free (identify avenues of free play)
  • Spending on play is not always a waste of money (have a budget for play)
  • We might not be physically closer to our friends/family a lot of times – hard to find a play partner
  • Lack of clarity on the difference between play, laziness and escapism from work.

How work and play can co-exist

  • Philosophy of play can be used in work also to some extent – lightness and freedom
  • Emphasise on quality of work, plan your work better and give space to serendipity
  • Identify the work-play pattern that suits you the best (most productive and most happy)
  • Happiness leads to productivity leads to happiness; Exhaustion leads to lower productivity leads to more exhaustion

Make play a habit (a necessity)

  • Shutdown regime (daily time off work – not just physically but mentally as well)
  • Meaningful play
  • Play requires discipline (prioritization, clarity)
  • Awareness at play, awareness at work (being present in the moment)
  • Design your play activities but not too much. Have specific non-negotiable slots in your day and week for fun and entertainment.
  • Have an open mind (stop judging yourself) – and dont feel guilty about having fun!
  • Take feedback from your mind (continuously check with your mind after each play activity)
  • Meditate!
  • Stop being paranoid about work once you have shut down
  • Find the kind of activities/experiences that are most enriching for you
  • Include human encounters

Creativity

“Creativity is just connecting things. When you ask creative people how they did something, they feel a little guilty because they didn’t really do it, they just saw something. It seemed obvious to them after a while. That’s because they were able to connect experiences they’ve had and synthesize new things. And the reason they were able to do that was that they’ve had more experiences or they have thought more about their experiences than other people.”

Steve Jobs, Founder of Apple & Pixar

Enhancing my creativity and thought process is something that has been on my agenda for a long time now. I commit to spend 45 minutes each day to try out an entirely new product for 15 days and review it every 15 days.

Open Science Hardware at GOSH 2018

Based on my work of supporting the maker movement in NIT Warangal through Innovation Garage [1], I was recently provided a full (travel + accommodation) scholarship to attend Gathering for Open Science Hardware (GOSH) in Shenzhen from 10th-13th October, 2018. This post is to share my learning about Open Science Hardware(OScH) movement and GOSH experience in general.

Understanding Open Science Hardware Through Examples

Example 1ย ย Imagine that you buy a 3D printer for your makerspace. You set it up and members start downloading/preparing STL files to print 3D models of their prototypes (and some cool jewellery too!). One day the printer stops working. ย Being a community makerspace, you canโ€™t afford to buy a new one altogether. But you and your friends have a maker mindset and are confident that you can fix the 3d printer instead of having to buy a new one. So cool! There is only one problem โ€“ the machine is โ€˜proprietaryโ€™ and there is no information available about how it has been designed and built. You are stuck.

Example 2ย ย You are a university student and have been studying about brain-computer interactions. You approach your professor with the idea of a project that requires an EEG device to record subjectโ€™s brain activity during gameplay. The professor is impressed but the project gets stalled since the lab would not agree to purchase such a costly device for a single project.

Example 3ย You love fitness bands but care about your privacy and fear that wearing a proprietary fitness band will put your data at risk. You would rather make your own fitness band if only the design details were openly available.

Example 4 ย You are an environment researcher who requires air pollution data of top 50 most populated cities in the world at different times of the day. However, your team is too small to record data at this scale and you decide to crowdsource the effort. Funding so many air quality monitors is not a viable option. You instead build a simple device using commonly available tools and document it in detail. Next, with the help of some international organizations, you reach out to high school students of these cities, remotely train them about how to build these monitors and are able to crowdsource data at a large scale. Not only that, you just made citizen scientists [2] out of high school kids of 50 cities! This is the power open source ๐Ÿ˜Š

To quote GOSH roadmap Executive Summary –

To summarize, the ability to use, study, replicate, and improve scientific instrumentation is a central part of experimental science, and plays a crucial role in public life, research, and action. However, these activities are currently restricted by proprietary instrumentation, which is difficult and expensive to obtain and maintain, since they cannot be fully inspected, evaluated, or customized. This situation is fundamentally detrimental to the production of knowledge and its potential for creating equitable and sustainable solutions. The Open Science Hardware (OScH) community therefore seeks to bring together developers and users of scientific tools and research infrastructures to support the pursuit and growth of knowledge through global access to hardware for science. [3]

 

What Happened at GOSH 2018?

GOSH 2018 was organised at Openfiesta, Tsinghua University, Shenzhen. Our aim was to come up with a list of action items based on the following three pillars of OScH community :

  • Learn about itself, the contexts in which it currently operates, and the ways in which OScH impacts society.
  • Support individuals and forge partnerships to create the conditions under which OScH can flourish
  • Grow with respect to local differences, increasing the diversity, scale and impact of the OScH community.

I have summarized our discussion on each of these here. (Since itโ€™s on the forum, you might be asked to join the forum to view it).

People from 40+ countries and multiple professional backgrounds (researchers, social scientists, artists, biologists, educators, entrepreneurs, hobbyists, etc.) came together for 4 days to discuss various issues related to open hardware like licensing, collaborations, educating, funding, citizen science, open science for UNโ€™s Sustainable Development Goals (SDGs) and so much more. More details can be found in the GOSH 2018 community report [4]. I saw some of the best makers and projects ever at GOSH. You can find the entire list of attendees on GOSH website to see if thereโ€™s someone that you would like to connect with. [5]

47682715_2062439987155934_1868804991351783424_o

Everyone @ GOSH 2018!

Post GOSH 2018, we are looking to leverage and empower local communities with support from global resources ultimately leading to global impact (Think global, Act local). Following are some of the action items that we came up with for India (or Asia in general). Please feel free to connect if you are interested in pushing forward any of these agenda:

  1. Organize AsiaOSH to drive the open hardware movement in Asia. (AfricaOSH is already happening this year in Tanzania!)
  2. Collaborate and prepare a map with all makerspaces of Asia.
  3. Identify low-cost alternatives for open hardware โ€“ map major suppliers, artisans and craftsmen of Your city (Hyderabad) to connect makers to suppliers.
  4. Organize a Repair day in your locality/makerspace (where people bring their broken stuff and tools and are guided on how to fix them)
  5. Collaborate with Indian universities to offer residential programs for Open Science community members working on important issues like soil, environment and climate.

Along with several brainstorming and unconference sessions, we also had several technical workshops – creating a particle accelerator, soil monitoring, air/water monitoring, cervix cancer detection, building DNA circuits, Microscopes, etc.

IMG_20181012_145745.jpg

Minerva and Juanma organizing workshop on How to code DNA (GOSH 2018)

We visited some of the coolest places in Shenzhen including a PCB manufacturing company, worldโ€™s first and largest hardware accelerator HAX, the enormous Huaqiangbei market, a social dinner by Kickstarter, X-factory and Public Lab barn raising. Some of us presented about GOSH at the Makerfaire organized by Shenzhen government in which I got a chance to introduce students to open hardware movement and speak about our topic sessions and corresponding action items.

43814675650_fcc7f46275_o.jpg

During panel discussion post my talk @ Makerfaire

 

Life in Shenzhen

I reached Shenzhen via the ferry from Hong Kong to Port Shekou. I was amazed to see the sky-high buildings and clean roads on my drive to the hotel.The gathering was in Tsinghua University which enabled me to live university life once again. Shenzhen is a heavenly destination if you are either a hardware junkie or a street-food lover. Make sure to try their street food, Fish Hot Pot and Bubble Tea. Donโ€™t be surprised if thereโ€™s meat inside some dessert.

And DO NOT, I repeat, DO NOT miss out on visiting the Huaqiangbei (pronounced whua-chung-bee) market [6]. In fact, spare an entire day (or two days if you can) only for it. There is no electronic part that you canโ€™t find here, if you know where to look. I realised what manufacturing at scale is only after I saw a complex with 100+ shops, all of which sold only one thing โ€“ iPhone and iPad cases.

45499979022_8ff23a3bec_o.jpg

A store in Huaqiangbee Mall

If you do not understand Mandarin and think that you can manage with English, you are in for a big surprise as very few people speak English there. I logged in to a desktop to type a document and could not use MS Word because everything was written in Mandarin! The restaurant menus, station names, sign boards, your hotel address and anything readable will most probably be only in Mandarin. So, make Google Translate your friend and make sure to install a solid VPN before reaching Shenzhen because several apps like Google, Whatsapp and Facebook are banned in China (there are alternatives and you can easily find that information online[7]). However, Chinese people are much like Indians โ€“ very helpful and offer great hospitality and you will do just fine with hand gestures even if the internet fails you.

Overall, it was an enlightening experience where I met people from all walks of life and different parts of the world, all united by one purpose โ€“ To make it easy for everyone to build their own prototyping and research equipment and usher an era of collaborative and open development in hardware, just like it has happened in software.

References

  1. IG impact analysis report for the year 2016-17
  2. Citizen Science : crowdsourcing for research
  3. Open science hardware roadmap
  4. GOSH 2018 community report
  5. GOSH 2018 participants
  6. Inside the worldโ€™s craziest gadget market by The Verge
  7. Apps that will help you in China

How we built the most creative hack of Arcesium Hackathon 2018

Note : This post is my collaborative effort withย Architย to document the development of AXL and help people trying a similar approach in of their solutions.

Arcesium recently organised a global hackathon that saw participation of ~50 teams firmwide. Excited to build something new, I jumped at the opportunity and teamed up with Ram Chidambaram, Archit Kansal and Shivaram Vasudevan.

We aimed to tackle a very common challenge – developers are not domain experts and domain experts do not have the technical skills to write code describing complex logic. This challenge is faced by almost all the companies that automate/simulate complex logic for their customers in a particular domain, e.g. insurance, billing, combat solutions, etc.

What is the most obvious solution? Give the domain experts the capability to read, understand (and with some skills, even write) the logic and let developers worry only about the extremely technical aspects. Enter Domain Specific Languages!

Why DSL

A DSL is a computer language specialized to a particular application domain. This is a contrast to general-purpose languages like C, Python and Java that are designed to let you write any type of program with any sort of logic you need.

The idea of a DSL(domain specific language) is nothing new and has been around for quite some time. However, the following are some of the ways in which we tried to make the AXL ( pronounced axle – Arcesium eXpression Language) stand out:

  • Simple, English-like syntax
  • Codebase size reduced by 80%
  • Time taken to write a new piece of code: 2-3 hrs (reduced from several days)
  • Gives users a lot more control through a UI
  • Reusability – Ability to write and import operators and modules written again in AXL

(more…)

32 Kms : My first leisure Bicycle ride

For the last eight months, I have been exploring ways to keep myself healthy, including workouts, nutrition, running and outdoor sports. This December, when I was looking for something new to do over the weekend, I came across a Facebook Event by The Bike Affair about a group bike ride to Osman Sagar.

The Bike Affair (TBA)

The Bike Affair is a company started by a bunch of biking enthusiasts that offer all bike-related services including rental, purchase, repair, demo, rides, etc. They organize multiple rides of various difficulty levels throughout the year. You can check out their upcoming rides and other services here.

 

No bike? No problem!

As I had never cycled for more than 5 kms in life, I was sure that this would be a challenging experience and I was super-excited. Tulika, my friend, joined as well. There was only one problem. We do not own bikes. So, we messaged TBA asking if they could provide us bikes on rent and they said that they could. The ride was on Sunday (Dec 17, 2017 to be precise). When me and Tulika reached their store on Sunday morning at 6 am, they already had our bikes ready. They adjusted the saddle heights according to our heights and showed us the basic gear controls as we had never rode a geared bike before. Meanwhile, other riders also joined us at the starting point (i.e. TBA Kondapur). We started at 6:30 sharp.

The route was going to be : TBA Kondapur – IIIT Junction – Movie Tower-Gandipet Lake (Osman Sagar) – and back. Our ride incharges were Ram and Ramesh from TBA.

(more…)

The Genesis of Smart News Scraper

I spent the last week coding away at Hackathon 5.0 with my friends Himanshu Garg and Priyanka Agrawal. It was organised by The Lakshya Foundation and Innovation Garage (the only place where I get to be the participant as well as the planner!) Here I discuss our problem statement, how we approached it and the end result.

The Problem Statement

As always, Hackathon offered quite a big and diverse list of interesting hardware and software problems. After much deliberation, we decided to work on Almabase’s problem which was to build a machine learning system to find news that is relevant to a particularย college. The problem statement consisted of two parts :

  1. Write crawlers to fetch news articles from a predefined list of news websites. Use RSS feeds if available.
  2. From the fetched articles,ย identify the ones that are relevant to a particular college. To begin with, consider only NIT Warangal. A list of synonyms of NITWย and an alumni database of names and batches will be provided to the teams.

We selectedย this problem because :

  1. We found it to be challenging. None of us had ever written a crawler orย applied ML and Data Classification concepts to practical problems. There was a lot of scope to learn.
  2. We use Almabase’s product NITWAA very often and we love it!
  3. We had heard a lot about the awesome team at Almabase from mutual friends and wereย eager to work with them ๐Ÿ™‚

Figuring It Out

We still had a week to go when the problem statement was released. Part 1 was relatively easy. We quicklyย learnt how basic crawlers are written and what are the various libraries available. We learnt aboutย several useful libraries like Beautiful Soup, Scrapy, Feedparser and Newspaper – all of which are simply amazing! By now, it was clear that we were going to build the application in Python on Django.

Now thatย we haveย all the content posted on various news websites, how do we classify it? Heck, how to even make our bot understand that if an article is about “Armstrong” – whether it is Neil Armstrong or the physical unit Armstrong or some alumnus of aย college Armstrong?ย We realized that we needed to take into account the context of the article e.g. if it says Armstrong ‘graduated from’ or ‘studied in’ , etc. we can say that the article is relevant to at least some college. This is where Natural Language Processingย came into picture and we were introduced to the mighty Natural Language Toolkit (NLTK)ย for python.

By going through a few blogs and the NLTK book, we formulated a solution. We will identify names of all the Persons and Organisations mentioned in the article and check if they exist in Almabase’s database. This is known as Named Entity Recognition. Cool, we got the keywords. Now how to use these keywords to classify the data as relevant or not? What exactly might be a helpful keyword apart from names of persons and organisations?

Asking For Help

We were clueless at this point given the wide variety of ML approaches and algorithms and our lack of experience with them. So I headed to the Systers mailing list and posted a question explaining the problem and my approach towards it. Surprisingly, I got a reply from Yoly Ceron in just a few hours (Thanks Yoly!) .

hack5

(more…)

Interview : Raghu @ Yellow Messenger

I recently interned at Yellow Messenger , an awesome startup working in conversational commerce space. It was an amazing experience and I got a chance to learn both technology and business/management aspects of the product. Here is my interview with Raghu Ravinutala, the CEO of Yellow Messenger.
raghu
Background
Raghu Ravinutala graduated from NIT Warangal in 2000 in Electronics and Communication Engineering. In his 15 year long career, he has worked at senior positions in companies like Texas Instruments and Broadcom.ย In 2014, he co-founded Yellow Messenger with Kishore Reddy (CSE, IIIT Hyderabad) – one of the very first conversational commerce platforms and is currently working full time as its CEO. Within one month of its launch, Yellow was selected as one of the top 10 startups in India by NextBigWhat and is one of the only four startups from India selected for the famous seed accelerator program, Startup Chile this year (which, by the way, happened before they had even started to work full time on it).

Raghu, you have led many significant projects during your 15-year long career. I would like to start by asking you about the one project that you consider to be most successful and why so?
The most successful project for me was our first startup called Telerad Tech, where we developed a product for medical images to be put on the cloud which could then be remotely accessed(through email) by both patients and concerned radiologists. This ruled out the need for the patient to collect their test reports personally and also facilitated to get a second opinion from other radiologists spread across globe. It was a completely new field for me, coming from a chip design background. I essentially did the project management for that, defined the product…and right now, itโ€™s being used by over 500 hospitals around the globe. The satisfaction that I get from this makes it the most successful product in my view.
Having led so many projects and teams, how would you crystallize your leadership mantra or the insights that you formed over the years?

(more…)

Hierarchy Generation Tool : Some Ideas

The Problem

Two of my friends and seniors, Sudheeshย S and Kranthi Kiran are working on an awesome project called Government Enhanced Administrative Resource (GEAR) systems. You can find the project on github. The idea is simple yet powerful. Suppose you are walking down a road and see an open manhole or a damaged road. All you need to do is take a picture and tweet it with your location. GEAR bots will identify your complaints from all the tweetsย and automatically lodge them with the concerned authorities. The status of various complaints is public. It improves the transparency of the system and the authorities are forced to act at the earliest.

They presented us with a unique and interesting problem inย the recent Hackathon – To create a hierarchy generation tool for GEAR systems so that the complaint can automatically go to the concerned authority and then automatically be movedย up or down the hierarchy if it is not addressed by the current assignee. The main challenge here is to build a generic GUI tool integrated with a smart databaseย table which can handle the different organizations and their hierarchies present inside theย GEAR Systems application. So, a simple tree…that’s it, right? The problem may look simple at the beginning but it really isn’t because :

  1. A hierarchy like ย Director > Deans > (Each dean further breaks down to others) > so on โ€ฆcreating a tree structure may or may not have cyclic dependencies making a partialย element of it a graph
  2. Multiple people can be present under multiple higher order authorities.
  3. There may be flat authority order that could be possible. (Dean 1 – Dean 2 – Dean 3)
  4. One person can have multiple different roles in different hierarchies. e.g. a professor can also be theย academic Dean.

Our approach

Following are the challenges that we identified at the beginning and wanted to tackle :

1. Huge Complexity :ย The hierarchies existing in the real-world can be very complex with several intersectionsย and overlaps. The difficulty increases exponentially with the size of the organisation.

2. Speed of information retrieval :ย There can be thousands of people and millions of connections in a hierarchy. Therefore,our solution should be efficient and memory management should be such that allows quickย data access and traversal along the hierarchy.

SOLUTION 1 :

We started by thinking at the lowermost level. (more…)