go_scholar

package module
v1.0.3 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Apr 21, 2024 License: Apache-2.0 Imports: 10 Imported by: 0

README

scholar

scholar is a WiP Go module that implements a querier and parser for Google Scholar's output. Its classes can be used independently, but it can also be invoked as a command-line tool.

This tool is inspired by scholar.py

Usage

import "github.com/compscidr/scholar"

sch := scholar.New()
articles := sch.QueryProfile("SbUmSEAAAAAJ", 1)

for _, article := range articles {
	// do something with the article
}

Features

Working:

  • Queries and parses a user profile by user id to get basic publication data
  • Queries each of the articles listed (up to 80) and parses the results for extra information
  • Caches the profile for a day, and articles for a week (need to confirm this is working)
    • This is in memory, so if the program is restarted, the cache is lost
  • Configurable limit to number of articles to query in one go

TODO:

  • Pagination of articles
  • Add throttling to avoid hitting the rate limit (figure out what the limit is)
  • Add on-disk caching so that if program restarts the cache is not lost

Possible throttle info:

https://stackoverflow.com/questions/60271587/how-long-is-the-error-429-toomanyrequests-cooldown

Documentation

Index

Constants

View Source
const AGENT = "Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Firefox/81.0"
View Source
const BaseURL = "http://scholar.google.com"
View Source
const MAX_TIME_ARTICLE = time.Second * 3600 * 24 * 7 // 1 week
View Source
const MAX_TIME_PROFILE = time.Second * 3600 * 24 // 1 day

Variables

This section is empty.

Functions

This section is empty.

Types

type Article

type Article struct {
	// contains filtered or unexported fields
}

func (Article) String

func (a Article) String() string

type Profile

type Profile struct {
	// contains filtered or unexported fields
}

type Scholar

type Scholar struct {
	// contains filtered or unexported fields
}

func New

func New() Scholar

func (Scholar) QueryArticle

func (sch Scholar) QueryArticle(url string, article Article, dumpResponse bool) Article

func (Scholar) QueryProfile

func (sch Scholar) QueryProfile(user string, limit int) []Article

func (Scholar) QueryProfileDumpResponse

func (sch Scholar) QueryProfileDumpResponse(user string, queryArticles bool, limit int, dumpResponse bool) []Article

QueryProfileDumpResponse queries the profile of a user and returns a list of articles if queryArticles is true, it will also query the articles for extra information which isn't present on the profile page

we may wish to set this to false if we are only interested in some article info, or we have a cache hit and we just
want to get updated information from the profile page only to save requests

if dumpResponse is true, it will print the response to stdout (useful for debugging)

func (Scholar) QueryProfileWithCache

func (sch Scholar) QueryProfileWithCache(user string, limit int) []Article

Directories

Path Synopsis
scholar module

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL