"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101 Firefox/102.0", // default user agent, currently firefox 102.0 on linux
@ -4,7 +4,7 @@ If you'd like to contribute your own code, first all we want to say thank you fo
First, we recommend learning to reverse engineer website in general before anything. This means knowing your way around DevTools (otherwise known as "Inspect Element"), among other factors of your browser and the site you're going to attempt writing code to bypass.
Specifically, you also should know how to write code in Node.js, and have experience with the `axios`, `puppeteer` and `cheerio` packages.
Specifically, you also should know how to write code in Node.js, and have experience with the `got`, `puppeteer` and `cheerio` packages.
## Guidelines for New Extractors
@ -12,6 +12,6 @@ The site must not already solveable under normal circumstances with BIFM. A good
## Guidelines for Updating Extractors
Your update must improve the extractor by either eliminating the need for a CAPTCHA solver, moving from Puppeteer to Axios, or simply fixing a bug.
Your update must improve the extractor by either eliminating the need for a CAPTCHA solver, moving from Puppeteer to got, or simply fixing a bug.
If you think you're ready, you can move onto the [basics of writing a bypass](./WRITING-EXTRACTORS.md)
@ -16,14 +16,14 @@ For example, if your module was going to be for a website called `link.com`, nam
## Choosing extractor type
When in doubt, use `axios` &`cheerio` over `puppeteer`. Puppeteer can be very useful for getting around sites that use CAPTCHAs or already have bypasses in [FastForward](https://fastforward.team), but Puppeteer can take up resources very quickly, as it's an entire browser window (hidden, of course) just to scrape a webpage.
When in doubt, use `got` &`cheerio` over `puppeteer`. Puppeteer can be very useful for getting around sites that use CAPTCHAs or already have bypasses in [FastForward](https://fastforward.team), but Puppeteer can take up resources very quickly, as it's an entire browser window (hidden, of course) just to scrape a webpage.
### Axios + Cheerio
### Got + Cheerio
Copy and paste this template into your extractor.
```js
const axios = require("axios");
const got = require("got");
const cheerio = require("cheerio");
const lib = require("../lib");
@ -32,7 +32,7 @@ module.exports = {
requiresCaptcha: false,
get: async function(url, opt) {
if (lib.config().debug == true) console.log("[scraper] Requesting page...");
let resp = await axios({
let resp = await got({
method: "GET",
url: url,
headers: {
@ -42,12 +42,12 @@ module.exports = {
});
if (lib.config().debug == true) console.log("[scraper] Got page. Parsing page...");
let $ = cheerio.load(resp.data);
let $ = cheerio.load(resp.body);
// do stuff with resp.data here
// do stuff with resp.body here
// or cheerio via $
let r = resp.data.split("whatever")[2]; // whatever your end result is, dont use this obviously though, it's an example
let r = resp.body.split("whatever")[2]; // whatever your end result is, dont use this obviously though, it's an example
|**`adf.ly`**|Axios|No|[http://usheethe.com/T3F5](http://usheethe.com/T3F5)|`usheethe.com`|There is not a complete list of domains under adf.ly's net that I could find, but there is an algorythm for finding adf.ly links and automatically bypassing it.|
|**`adf.ly`**|Got|No|[http://usheethe.com/T3F5](http://usheethe.com/T3F5)|`usheethe.com`|There is not a complete list of domains under adf.ly's net that I could find, but there is an algorythm for finding adf.ly links and automatically bypassing it.|
|**`adlinkfly`**|Puppeteer + Stealth|Sometimes, but always required|[https://pdiskshortener.com/6I2CR2](https://pdiskshortener.com/6I2CR2), [https://dz-linkk.com/N2xFP](https://dz-linkk.com/N2xFP), [https://www.shortly.xyz/r/74fee2919710769f09131da1beb8243e](https://www.shortly.xyz/r/74fee2919710769f09131da1beb8243e)|`adlinkfly.mightyscripts.xyz`, `pdiskshortener.com`, `dzlinkk.com`|Like adf.ly, there is not a list of adlinkfly sites, but there is an algorithm to find and bypass them.|
|**`aylink.co`**|Puppeteer + Stealth|Yes|[https://aylink.co/xay664](https://aylink.co/xay664) redirects to `https://mega.nz/folder/hpIAnRLS#fJZX651qRZdf_DU2Y_k0kA`||
|**`bc.vc`**|Puppeteer + Stealth|No|[https://bc.vc/vQesLIh](https://bc.vc/vQesLIh) redirects to `https://universal-bypass.org/`.||
|**`DaddyScripts`**|Axios|Sometimes, not always required|[https://lnk.snahp.eu/EBRs14BxyZ5oIlvrEx4EZGPlrwGm31VeNZ1Uo](https://lnk.snahp.eu/EBRs14BxyZ5oIlvrEx4EZGPlrwGm31VeNZ1Uo) (no password), [https://lnk.snahp.eu/W4hPgdadk7F9WtY5hb00uCdWzHZK5n5kt32ZG](https://lnk.snahp.eu/W4hPgdadk7F9WtY5hb00uCdWzHZK5n5kt32ZG) (password: `bifm`)|||
|**`DaddyScripts`**|Got|Sometimes, not always required|[https://lnk.snahp.eu/EBRs14BxyZ5oIlvrEx4EZGPlrwGm31VeNZ1Uo](https://lnk.snahp.eu/EBRs14BxyZ5oIlvrEx4EZGPlrwGm31VeNZ1Uo) (no password), [https://lnk.snahp.eu/W4hPgdadk7F9WtY5hb00uCdWzHZK5n5kt32ZG](https://lnk.snahp.eu/W4hPgdadk7F9WtY5hb00uCdWzHZK5n5kt32ZG) (password: `bifm`)|||
|**`fc.lc`**|Puppeteer + Adblocker + Stealth|No|[https://fc-lc.com/mWGCzBwu](https://fc-lc.com/mWGCzBwu)|`fc-lc.com`|Not officially a seperate extractor, this uses the `exeio` extractor.|
|**`gplinks.co`**|Puppeteer + Adblocker + Stealth|Yes||`gplinks.in`|Need example link that is SFW and legal.|
|**`karung.in`**|Axios|No|[http://karung.in/Gyucc](http://karung.in/Gyucc) redirects to `https://drive.google.com/uc?id=0B263gKU-C09_WW5rbURLeXN5QXc&export=download`.||Passworded links are currently not supported.|
|**`karung.in`**|Got|No|[http://karung.in/Gyucc](http://karung.in/Gyucc) redirects to `https://drive.google.com/uc?id=0B263gKU-C09_WW5rbURLeXN5QXc&export=download`.||Passworded links are currently not supported.|
|**`keeplinks.org`**|Puppeteer + Adblocker + Stealth|No|[https://www.keeplinks.org/p100/62b878489fbc5](https://www.keeplinks.org/p100/62b878489fbc5)||Passworded links don't need a password due to a cookie exploit.|
|**`linkvertise.net`**|Axios|Yes|[https://linkvertise.com/425581/example1](https://linkvertise.com/425581/example1) for regular redirects, [https://link-target.net/425581/example](https://link-target.net/425581/example) for a paste|`linkvertise.com`, `up-to-down.net`, `link-to.net`, `direct-link.net`, `linkvertise.download`, `file-link.net`, `link-center.net`, `link-target.net`, `link-hub.net`||
|**`linkvertise.net`**|Got|Yes|[https://linkvertise.com/425581/example1](https://linkvertise.com/425581/example1) for regular redirects, [https://link-target.net/425581/example](https://link-target.net/425581/example) for a paste|`linkvertise.com`, `up-to-down.net`, `link-to.net`, `direct-link.net`, `linkvertise.download`, `file-link.net`, `link-center.net`, `link-target.net`, `link-hub.net`||
|**`link1s.com`**|Puppeteer + Adblocker + Stealth|No|||Need example link that is SFW and legal.|
|**`sh.st`**|Axios|No|[http://ceesty.com/es47QR](http://ceesty.com/es47QR)|`ceesty.com`, `cestyy.com`, `clkme.me`, `clkmein.com`, `cllkme.com`, `corneey.com`, `destyy.com`, `festyy.com`, `gestyy.com`, `jnw0.me`, `xiw34.com`, `wiid.me`|The list to the right may not be 100% correct or complete.|
|**`sh.st`**|Got|No|[http://ceesty.com/es47QR](http://ceesty.com/es47QR)|`ceesty.com`, `cestyy.com`, `clkme.me`, `clkmein.com`, `cllkme.com`, `corneey.com`, `destyy.com`, `festyy.com`, `gestyy.com`, `jnw0.me`, `xiw34.com`, `wiid.me`|The list to the right may not be 100% correct or complete.|
|**`srt.am`**|Puppeteer|No|[http://srt.am/e8kZ9m](http://srt.am/e8kZ9m) redirects to `https://www.google.com/`|`short.am`||
|**`tei.ai`**|Axios|No|[https://tei.ai/HOI4_1_11_11](https://tei.ai/HOI4_1_11_11) redirects to `https://www.mediafire.com/file/mmxskm3b1uanzfn/OG9134-HOI4FU1.rar/file`|`tii.ai`||
|**`thinfi.com`**|Axios|No|[https://thinfi.com/088ud](https://thinfi.com/088ud) (no password), [https://thinfi.com/088uk](https://thinfi.com/088uk) (password: `bifm`)||
|**`tei.ai`**|Got|No|[https://tei.ai/HOI4_1_11_11](https://tei.ai/HOI4_1_11_11) redirects to `https://www.mediafire.com/file/mmxskm3b1uanzfn/OG9134-HOI4FU1.rar/file`|`tii.ai`||
|**`thinfi.com`**|Got|No|[https://thinfi.com/088ud](https://thinfi.com/088ud) (no password), [https://thinfi.com/088uk](https://thinfi.com/088uk) (password: `bifm`)||
|**`try2link.com`**|Puppeteer + Adblocker|No|||Need example link that is SFW and legal.|
|**`wpsafelink`**|Axios|No|**Regular WPSafelink**: [https://demo-safelink.themeson.com/template1/?f7fbb8af](https://demo-safelink.themeson.com/template1/?f7fbb8af) redirects to `https://themeson.com/safelink/`**Long WPSafelink**: [https://pdiskshortener.in/svJ1](https://pdiskshortener.in/svJ1)|`demo-safelink.themeson.com`||
|**`wpsafelink`**|Got|No|**Regular WPSafelink**: [https://demo-safelink.themeson.com/template1/?f7fbb8af](https://demo-safelink.themeson.com/template1/?f7fbb8af) redirects to `https://themeson.com/safelink/`**Long WPSafelink**: [https://pdiskshortener.in/svJ1](https://pdiskshortener.in/svJ1)|`demo-safelink.themeson.com`||
throw"Boostme.link bypass has changed or we have been rate limited. If you are the owner of this instance, please dump the terminal contents into an issue on the repo.";
