Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Appearance settings

Add image files ending with star (*) instead of specifying the format #217

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Dec 7, 2019

Conversation

jjcaballero
Copy link
Contributor

Some context:
I was trying to include plots generated by matplotlib.sphinext when using the confluence builder. Then I learned that plots were properly generated and the only problem was the matplotlib.sphinext extension was not considering confluence as a possible builder.
Therefore, I created this PR (matplotlib/matplotlib#14681) there, to ask them to include it. Eventually, this was addressed in other PR (matplotlib/matplotlib#14683) which enhance the same but in a more general way. Now, builders which want to include plots from matplotlibs should handle images ending with star.

And handling image with star is the goal of this PR. I have added the parameter supported_image_types to the builder like here: sphinx builders doc
We might want to include more image formats, I have just considered png, jpeg and pdf.

In a nutshell:
when the image ends with star and does not exists, then the ConfluenceAssetManager will use the first found image with the same name but ending with a supported image type.

Side note: maybe I am wrong, but I think there are no tests to verify the publishing itself. Is there any recommendation to do such a thing? It would be very convenient to be able to run tests choosing confluence server and parent page. I mean using the same infrastructure used to run tests but publishing.

@jdknight
Copy link
Member

jdknight commented Oct 12, 2019

To start off, it is most likely that #228 will be merged in before this change (which will cause conflicts; but I do not mind correcting these conflicts if needed).

I have no problem with the idea of supporting a wildcard to map to detected assets which may be generated by other extension. I would like to also try this out to actually see it being performed (even though the idea makes sense after reading the referenced PRs). I am a bit hesistent in having this supported_image_types list though. Sphinx does have some mime definitions inside their images.py implementation -- I wonder if there is something we could be doing with this to prevent this extension to trying to manage supported image types (although, I have no problem definition a separate independent list to whitelist or blacklist image types based off official documented Confluence supported image types).

Note that due to other issues being addressed in this extension, the expected window for this feature is planned around end-of-year (2019; see this comment for more details).

@jdknight jdknight added the enhancement Something that can better improve this extension label Oct 12, 2019
@jessetan
Copy link
Contributor

jessetan commented Nov 6, 2019

I have a similar usecase. I build both HTML and PDF output from a single source. To make images (especially diagrams) display as crisp as possible, I use SVG diagrams for HTML output and PDF diagrams for inclusion into PDF output.
The image is included using .. figure:: images/diagram.*. The actual file (diagram.svg or diagram.pdf) that is included depends on the builder that is used.

Sphinx already has code that handles an asterisk in the file extension: https://github.com/sphinx-doc/sphinx/blob/01f8dad160d206a48302b1bcd91b4e0bc36e8a10/sphinx/builders/__init__.py#L186. This replaces the * in the filename based on the supported_image_types of a writer.
This is called by the default writers in Sphinx but not by confluencebuilder.I can make my usecase work by adding a suitable supported_image_types and adding a call to self.post_process_images(doctree) in prepare_writing() here:


(The exact line number is not important, but it needs to run on every documents doctree before the call self.assets.process(ordered_docnames).

@jjcaballero would that solve your usecase? It is much less code than this PR and will keep maintenance of the code that finds the correct file the responsibility of Sphinx, not confluencebuilder.

I wonder if there is something we could be doing with this to prevent this extension to trying to manage supported image types

@jdknight As Confluence content ends up as HTML anyway, one could assume that the image formats that are supported are as the same for the HTML builder. The list of supported images can then be defined in code with:

from sphinx.builders.html import StandaloneHTMLBuilder 

...

class ConfluenceBuilder(Builder):
    supported_image_types = StandaloneHTMLBuilder.supported_image_types

@jjcaballero
Copy link
Contributor Author

@jessetan your approach seems cleaner and works for my use case too.
Do you plan to do a PR for that or should I include it here?
I would like to try to keep the expected window for this feature mentioned by @jdknight (end of the year 2019)

@jessetan
Copy link
Contributor

@jjcaballero if you are willing, it would be great if you could make the changes in this PR. It would save me some work and you have the testcases ready.

After my previous comment, I've found the best place to call self.env.app.builder.post_process_images(doctree) is before processDocument in

self.processDocument(doctree, docname)

So far I have found only one downside; builds fail if the PlantUML Sphinx extension is used (https://github.com/sphinx-contrib/plantuml/). The image nodes inserted by that plugin never get a candidates attribute.
I've worked around this by adding the following before the call to post_process_images

        # Verify every image has a 'candidates' attribute with key '*'
        # This is normally added by Sphinx' ImageCollector, but is not called
        # by sphinxcontrib.plantuml for some reason
        # https://github.com/sphinx-doc/sphinx/blob/089046979f2b84cf850ee95776d10fe9fd9a77e9/sphinx/environment/collectors/asset.py#L43
        for node in doctree.traverse(imageNode):
            if 'candidates' not in node:
                node['candidates'] = { '*': node['uri'] }

There must be a better way to do this, but I haven't had time to delve into this issue any deeper. It's probably not necessary to try to fix that in this PR.

@jjcaballero jjcaballero force-pushed the handle-image-with-star branch from b01eacc to 794a92f Compare November 21, 2019 21:01
@jjcaballero
Copy link
Contributor Author

@jdknight the pull request should be ready now to review/merge. Thanks.

@jessetan eventually, I placed self.post_process_images(doctree) in prepare_writing() from builder.py. As this method belongs to the parent class sphinx.Builder, IMO it is clearer to call it directly from the child class ConfluenceBuilder, instead of using the self.env.app.builder from the assets.py module.

I agree that the plantuml issue does not need to be addressed here and I hope it does not stop this PR :-)

@jessetan
Copy link
Contributor

@jjcaballero this works for my usecase. Thanks for your effort.

@jdknight jdknight added this to the 1.2 milestone Nov 23, 2019
test/unit-tests/common/test_image_star.py Show resolved Hide resolved
test/unit-tests/common/test_image_star.py Outdated Show resolved Hide resolved
test/unit-tests/common/test_image_star.py Outdated Show resolved Hide resolved
test/unit-tests/common/test_image_star.py Outdated Show resolved Hide resolved
sphinxcontrib/confluencebuilder/builder.py Outdated Show resolved Hide resolved
test/unit-tests/common/test_image_star.py Outdated Show resolved Hide resolved
jjcaballero and others added 2 commits December 7, 2019 16:21
Signed-off-by: jjcaballero <jjcaballerofernandez@gmail.com>
[squashed and re-write commit message]
Signed-off-by: James Knight <james.d.knight@live.com>
Confluence builder's recent support for automatic image detection
(asterisk; [1]) adjusts the builder to use the common
`post_process_image` method to manage these image types. This will
result in the image post-transform to download image assets when
building. This is not necessary for Confluence builder as Confluence
instances support remote images. Adjusting the builder's default
configuration to reflect this.

[1]: f19633a

Signed-off-by: James Knight <james.d.knight@live.com>
@jdknight jdknight force-pushed the handle-image-with-star branch from 4002c76 to 2d6db0b Compare December 7, 2019 21:30
@jdknight jdknight self-assigned this Dec 7, 2019
@jdknight jdknight requested review from jdknight and removed request for jdknight December 7, 2019 21:30
With changes to the builder for image processing [1], the base builder
implementation requires the `candidates` attribute for images to already
be populated; adjusting.

[1]: f19633a

Signed-off-by: James Knight <james.d.knight@live.com>
@jdknight jdknight merged commit 1729c3b into sphinx-contrib:master Dec 7, 2019
@jdknight
Copy link
Member

jdknight commented Dec 7, 2019

Go time; thanks all! Merged into master; available next stable.

@jdknight jdknight removed their assignment Dec 7, 2019
@jjcaballero jjcaballero deleted the handle-image-with-star branch December 8, 2019 15:10
jdknight added a commit that referenced this pull request Dec 24, 2020
When populating the builders `supported_image_types`, the original
implementation would use a copy of the `StandaloneHTMLBuilder` builder's
supported images [1]. However, Confluence does support a different
subset of image types, so it is most likely better to define an explicit
list which works for both the existing Sphinx engines support and the
target Confluence instances supported (counter to the original review
comments).

This commit introduces `SUPPORTED_IMAGE_TYPES`, a list of mime types
which are registered in the Confluence implementation which also work
with Sphinx. The only change from using `StandaloneHTMLBuilder`s set of
images types to this list, is the added support for BMP files
(`image/x-ms-bmp`).

To provide a bit more flexibility, depending on newer versions of Sphinx
or unique Confluence instance support, the configuration option
`confluence_additional_mime_types` is being introduced to allow users to
extend the supported types if needed.

[1]: #217

Signed-off-by: James Knight <james.d.knight@live.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Something that can better improve this extension
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants
Morty Proxy This is a proxified and sanitized view of the page, visit original site.