Need advice on Web Crawling based Search App academic project

2021.10.21 12:48 Neerd7 Need advice on Web Crawling based Search App academic project

Hello, everyone. Might be slightly a long read, too much information to process.
As the title suggests, the project that my team of 3 members was assigned is for a junior year level Software Engineering course. A bit overwhelmed and confused on where do we actually start to make the whole thing as none of us have first hand experience with web tech and there's an abundance of resources. Any suggestions are appreciated, so here's what the project will do:

  1. For the web browser part: There will be an user side of things and an admin side of things (duh). The users should be able to sign up with their Google accounts.
Then, the users can create and use search clusters(one or multiple), where they will be able to provide a list of URLs as input, and what type of textual files like (any textual content, PDFs, docx, txt) they will also be able to provide these as inputs. So the user will be configuring the input URLs and what type of file they'd like. For the searching part the users will also be able to provide the search text for which they wish to get the results from. The users should also be able to provide a max depth of how deep we should search jnside the given url.
Now, as output we need to- we need to show the resultant links and based on the search text we need to some contextual text, some text infront or behind the matching search text (just like google/other search engines). We will have to index the results inside a database. The search results would be a paginated output with the user getting an email notification when it's done.
  1. For the android app part: The users would be able to create the search texts to search from and select the clusters. Only one added thing would be they'd be able to do these with voice commands.
That's about it for the requirements. Now how do we begin to tackle each of the steps? The faculty suggested to use Apache Lucene because it'd be useful for storing user information, admin information, cluster metadata, and the resultant data indexing. But one thing is bugging me is that it looks like really old and not that many learning resources available online nor anybody is actively using it enough to be available for support when we get stuck...
I've researched a lot and saw stuff like python libraries which are Selenium, beautifulsoup, would they be useful for our case? We intend to use Django for most of the backend stuff like signup using google as using one server side framework is a must.
Need help and nods in the right direction on how do we proceed and solve this project... thanks for having the patience to read, really really need suggestions. pls don't brutally downvote if i said something dumb, correct me instead.
submitted by Neerd7 to AskProgramming [link] [comments]


2021.10.21 12:48 veganmark Couple Diagnosed With Heart Injuries Shortly After Receiving The Pfizer COVID-19 Vaccine

Couple Diagnosed With Heart Injuries Shortly After Receiving The Pfizer COVID-19 Vaccine submitted by veganmark to WayOfTheBern [link] [comments]


2021.10.21 12:48 pokemastertails I can't load into most of my matches, is this happening with anyone else?

So I've logged on today and tried to play the new Halloween, revamped map. I've played 12 games and 9 of them, I was stuck on the loading screen. I'm not sure if it's just my internet or if this is happening with anyone else.
submitted by pokemastertails to PokemonUnite [link] [comments]


2021.10.21 12:48 vansh8138 Doea pi 400 support usb headphones audio output

Can i use usb headphones(redgear cosmo 7.1) with raspberry pi 400 Help me please i am unable to do my studies my laptop is gone and now i would only buy Raspberry pi 400 only when it supports my Headphones
submitted by vansh8138 to RaspberryPi400 [link] [comments]


2021.10.21 12:48 EBofEB Poor audio quality for meeting on Peoples Gas site frustrates listeners; many criticize proposal as too auto-centric - Nadig Newspapers - Northwest Sid

Poor audio quality for meeting on Peoples Gas site frustrates listeners; many criticize proposal as too auto-centric - Nadig Newspapers - Northwest Sid submitted by EBofEB to greatNWside [link] [comments]


2021.10.21 12:48 Sea-Rent7106 Viejo proverbio chino dice:

Viejo proverbio chino dice: submitted by Sea-Rent7106 to Carola [link] [comments]


2021.10.21 12:48 Hoggy2099 Get Relax Mom

Get Relax Mom submitted by Hoggy2099 to socialmilf [link] [comments]


2021.10.21 12:48 ChampionshipNice9211 My grades are...

I have all A's, 95-100 A's in all my classes except one. I have one high C(78) and I'm bit worried about that. I hope I can get it to an A too but Chemistry is a bit hard for me.
submitted by ChampionshipNice9211 to highschool [link] [comments]


2021.10.21 12:48 InterviewKey3451 Oculus 2 charging

My cat ate my charger so can i use any charger or does it have to be the official cord
submitted by InterviewKey3451 to OculusQuest [link] [comments]


2021.10.21 12:48 apdaga Handwriting Analysis #1: [Base of the Personality] (1/3) Emotionally Withdrawn | APDaga

Handwriting Analysis #1: [Base of the Personality] (1/3) Emotionally Withdrawn | APDaga submitted by apdaga to APDaga [link] [comments]


2021.10.21 12:48 siloowns Looking for atlas strat

Hey guys. Hope everyone is doing well and excited for Scourge league! So in previous leagues, when we got the watch stones the go-to atlas strat was the leapfrog strat. With there now only being 4 regions, does anyone have or is there an optimal strat for the atlas? If so can you please share, or suggest any help. Thanks and I hope RNGeezus is on yourside. See you in wraeclast!
submitted by siloowns to pathofexile [link] [comments]


2021.10.21 12:48 ryanxcore [WTS] Terre d’Hermes EDT 100ml (bottle)

https://i.imgur.com/W4YQKCl.jpg
FOR SALE: * Hermès - Terre d’Hermes (100ml/95%) $45

Thanks! I have a few more listings going up, please check my other posts if you’re interested in more of the same.
submitted by ryanxcore to fragranceswap [link] [comments]


2021.10.21 12:48 SwellyTR NEXT GEN. - A Playlist For Upcoming Rap Artists (Trap, Rage, Virtual, Experimental, Hyperpop)

NEXT GEN. - A Playlist For Upcoming Rap Artists (Trap, Rage, Virtual, Experimental, Hyperpop) submitted by SwellyTR to ProjectListen [link] [comments]


2021.10.21 12:48 skridalthcrevasse Chills

Chills submitted by skridalthcrevasse to meme [link] [comments]


2021.10.21 12:48 BinaryOptionAlliance FlokiKiller🐕 | Next x100 gem 💎 | Flokis last wish 💭| huge marketing 📈| FairLaunch 💰| Busd reward 🤑

Welcome to Flokikiller🐶 The last wish from Floki! 💸BUSD Reward Token |Launch 26. October
We have set ourselves the goal of making Floki’s last wish come true, be part of our journey and let’s make FlokiKiller 🐶 one of the most successful coins in token history together.
💸 All holders of $FlokiKiller tokens automatically receive $BUSD tokens airdropped into their wallets every 1 hours
💰tokenomics 🏦Supply: 100,000,000,000 total 🛒Max buy/sell: 1% 👝Max wallet: 2% 🤑Rewards: 5% BUSD $ 🗽Marketing: 6% 🔥 Burn: 30% 🔐lq locked
🐳 Anti-Whale and Anti-Bot Features: Maximum wallet cap and maximum transaction amount of 2% of supply
We plan a Huge marketing, which includes ➡️ ads on Poocoin ➡️ ads on Coinsniper ➡️ popular crypto influencer ➡️ listings ➡️ CG ➡️ CMC
see you on Telegram your FlokiKiller Team
TG: https://t.me/FlokiKillerOfficial Website: https://www.flokikiller.com
submitted by BinaryOptionAlliance to MarsWallStreet [link] [comments]


2021.10.21 12:48 nem_v_39 Highlights from Get up! Shout!

submitted by nem_v_39 to ShamanKing [link] [comments]


2021.10.21 12:48 ChromeSphexYT Overwatch Halloween Terror 2021 in a nutshell

Overwatch Halloween Terror 2021 in a nutshell submitted by ChromeSphexYT to YouTubeGamers [link] [comments]


2021.10.21 12:48 YesDatGuy I'm so lucky

Hey, everyone just wanted to show my affection for the gem I have in the form of my girlfriend. She has been suffering from BPD and for the last 4 years we have been together in a long distance relationship. Every night we keep talking till one of us feels too sleepy. Last night I fell asleep accidentally while talking because I was very tired from work. She kept calling me throughout the night worried as hell but I didn't wake up. I woke up in the morning and realized how badly I had messed up. But instead of being upset at me she was genuinely glad that I was okay. She helps me a lot too whenever she can (I suspect I have ADHD but have never been diagnosed). She's such a great person and makes me feel lucky everyday
submitted by YesDatGuy to happyrelationships [link] [comments]


2021.10.21 12:48 aeghn I made a fork that supports the bottom titlebar and displays the active window title and icon in the i3bar

I made a fork that supports the bottom titlebar and displays the active window title and icon in the i3bar submitted by aeghn to i3wm [link] [comments]


2021.10.21 12:48 Graptemys14 I’m making $28.50 at my new job. Is that a good price for these economic times?

Give me feedback from where you are economically so I can see if this is a good wage. Examples: Demographic, position, experience, etc.
submitted by Graptemys14 to jobs [link] [comments]


2021.10.21 12:48 WiseGuy1882 ‘The Croods’ Berlin Premiere 15th February 2013

‘The Croods’ Berlin Premiere 15th February 2013 submitted by WiseGuy1882 to EmmaStone [link] [comments]


2021.10.21 12:48 d871649d Figures

Figures submitted by d871649d to arttocope [link] [comments]


2021.10.21 12:48 StevynTheHero I'm on a losing streak for the same basic reason

As a defender, I take the role to disrupt the enemy team. Stun them, displace them, interrupt them, etc etc.
So why, oh team mates of mine, do you take the golen opportunities and amazing openings I create for you, and waste them by attacking Drednaw/Zapdos directly when the entire enemy team is here, and ready to snipe the killing blow, making all our efforts for naught? Wouldn't it be better to fight them, wipe them, and then SAFELY obtain the objective?
How many times must this happen before you learn?
Why should I even play a defender if you're not going to utilize my role properly?
What can I do to make you see?
submitted by StevynTheHero to PokemonUnite [link] [comments]


2021.10.21 12:48 Ve8rel Lets go bois

Lets go bois submitted by Ve8rel to gaming [link] [comments]


2021.10.21 12:48 ReviewEquivalent1266 Immigration judges, lawyers, and visitors must be vaccinated but Biden has exempted all illegal aliens from vaccine mandates. Furthermore, illegal aliens don't even have to show negative Covid-19 tests to enter federal buildings as American citizens must.

submitted by ReviewEquivalent1266 to Deplatformed_ [link] [comments]


http://salon-victoriya.ru