Replies: 3 comments · 5 replies
-
|
Did you solve it? I also encountered the same problem. |
Beta Was this translation helpful? Give feedback.
-
|
I didn't dig into the code, so no. I hoped that one of the devs could take a look. |
Beta Was this translation helpful? Give feedback.
-
|
I have currently written a program to solve this problem |
Beta Was this translation helpful? Give feedback.
-
Are you able to share, also facing this issue as I need the pure XML returned from an RSS feed. |
Beta Was this translation helpful? Give feedback.
-
|
I can't guarantee that the format processed by the program is correct. I only tested it on two RSS websites. package main
import (
"bytes"
"encoding/json"
"fmt"
"github.com/antchfx/htmlquery"
"github.com/antchfx/xmlquery"
"github.com/gin-gonic/gin"
"html"
"io"
"net/http"
"strings"
)
type Response struct {
Status string `json:"status"`
Message string `json:"message"`
Solution struct {
Response string `json:"response"`
Headers map[string]string `json:"headers"`
} `json:"solution"`
}
func main() {
app := gin.Default()
app.GET("/*url", func(c *gin.Context) {
url := c.Param("url")[1:]
urlWithQuery := url + "?" + c.Request.URL.RawQuery
payload := []byte(fmt.Sprintf(`{"cmd": "request.get", "url": "%s"}`, urlWithQuery))
buffer := bytes.NewBuffer(payload)
post, err := http.Post("http://localhost:8191/v1", "application/json", buffer)
if err != nil {
print(err)
return
}
defer post.Body.Close()
all, err := io.ReadAll(post.Body)
if err != nil {
print(err)
return
}
var response Response
err = json.Unmarshal(all, &response)
if err != nil {
c.String(200, string(all))
return
}
if response.Status != "ok" {
c.String(500, response.Message)
return
}
c.Header("Content-Type", response.Solution.Headers["content-type"])
if strings.Contains(response.Solution.Response, "<body><div id=\"webkit-xml-viewer-source-xml\">") {
doc, err := xmlquery.Parse(strings.NewReader(response.Solution.Response))
if err != nil {
return
}
node := xmlquery.FindOne(doc, "//div[@id=\"webkit-xml-viewer-source-xml\"]")
c.Header("Content-Type", "text/xml;charset=UTF-8")
c.String(200, node.OutputXML(false))
return
}
if strings.Contains(response.Solution.Response, "?xml version") {
if strings.Contains(response.Solution.Response, "<body><pre") {
root, err := htmlquery.Parse(strings.NewReader(response.Solution.Response))
if err != nil {
c.String(500, err.Error())
return
}
c.Header("Content-Type", "text/xml;charset=UTF-8")
c.String(200, html.UnescapeString(htmlquery.OutputHTML(htmlquery.FindOne(root, "//pre"), false)))
return
}
}
c.String(200, response.Solution.Response)
})
app.Run(":8192")
} |
Beta Was this translation helpful? Give feedback.
-
|
@gl0zzy thank you, I see how it works (just not a coder myself)... on that topic I'm also running my FlareSolverr in a Docker container on a NAS (pretty much not touched the code and dumbly getting my results via a remote CURL to that NAS IP on port 8196. Any quick nudge in the direction I'd have to take to get you addition above running and then listening to 8192 with such a setup (consider me a total noob on non-html code and a person mainly using GUI interfaces... any details that might help me get going in the right direction) :) |
Beta Was this translation helpful? Give feedback.
-
|
https://github.com/gl0zzy/flaresolverr-proxy |
Beta Was this translation helpful? Give feedback.
-
|
Thank you, let me see how far I get :) |
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
-
Hi, I'm trying to fetch an RSS feed which is behind Cloudflare
Unfortunately in the response it's not the raw XML but the rendered source code:
I've tried the
returnRawHtmloption but didn't change anything.Is there a way to get the real feed source code?
it should start with:
Thank you in advance.
Beta Was this translation helpful? Give feedback.
All reactions