Add top-level Text function for BeautifulSoup-style text extraction - #596
Open
ChrisJr404 wants to merge 2 commits into
Open
Add top-level Text function for BeautifulSoup-style text extraction#596ChrisJr404 wants to merge 2 commits into
ChrisJr404 wants to merge 2 commits into
Conversation
Adds goquery.Text(s, *TextOptions), a package-level counterpart to the Selection.Text method that gives control over how the text of distinct text nodes is joined (Separator), whether each node is trimmed of surrounding whitespace (Trim), and which text nodes are included (Keep). This covers the common need to extract clean, readable text from a document - for example joining fragments with a space and dropping the text of script/style elements - without hand-rolling a node walk. Passing a nil TextOptions keeps the behaviour identical to Selection.Text. Includes tests and a runnable example.
Member
|
Hello Chris, Thanks for this PR! Took a quick glance this morning and it looks great, I will wait to merge for when I have a bit more time to dive into it. In the meantime, I noticed some edge cases that should probably be covered by a test, if you have a moment to update it - I'm thinking of calling Thanks again, |
Author
|
Added the two cases you asked for, empty selection and everything filtered out by Keep. Thanks, Martin. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This adds
goquery.Text(s *Selection, opts *TextOptions), a top-level counterpart to theSelection.Textmethod that gives control over how text is extracted, so callers can get clean, readable text out of a document without hand-rolling a node walk.This is the feature discussed in #443, where the request was to offer something similar to Python BeautifulSoup's
get_text. As suggested there, it is a package-level function rather than aSelectionmethod (to keep the method set aligned with the jQuery API, likeNodeNameandOuterHtml), and it is "general enough" via a small options struct:Separatoris inserted between the contents of consecutive text nodes.Trimstrips leading/trailing whitespace from each text node and drops the ones that become empty (the insignificant whitespace from source indentation).Keepis an optional predicate called per text node; returning false excludes it, which is how a caller drops the text of<script>/<style>elements by inspecting the node's parent.A
nil(or zero-value)TextOptionsmakesTextbehave exactly likeSelection.Text, so the default is unsurprising and the traversal semantics match the existing method.Example:
Tests cover the nil-options equivalence to
Selection.Text, the separator/trim behaviour, theKeepfilter, and a multi-node selection; a runnableExampleTextis included as documentation.go test ./...,go vet ./..., andgofmtare all clean.Closes #443.