This is just a try Infra asked about. I have also asked montastic if we can
have 3 different badges. For now I simply backport this one in order to test
all the Buildbot flows.
In previous commit I used Gradle application plugin and removed the
jvmArguments var used below in build.gradle. This restores jvmArguments, fixes
a problem with mainClassName missing in application plugin, simplifies
applicationDefaultJvmArgs and move ext.ofbizMainClass to a more logical place
The previous commit was twice wrong:
1. System properties in gradle.properties are not defined using -D but using
systemProp.
2. Anyway systemProp. is defining system properties only available in JVM where
Gradle is running, not the application you run. For that you need to use
applicationDefaultJvmArgs in application in the main build.gradle.
Here is the system property for jdk.serialFilter
Conflicts handled by hand in build.gradle
When importing an entity with "${" in for at least an element it's rejected
because of the security check done to protect from Freemarker unauth attacks
(see OFBIZ-12594).
As suggested by Ingo, allowing users with appropriate permissions seems an
usable solution. We still need to define the "appropriate permissions".
We can start with OFBTOOLS and WEBTOOLS, as it's reported by Ingo, and add
others later if they ever come.
Thanks: Ingo Wolfmayr for report and suggestion
It's now 2 weeks that INFRA-23076 is open, so like 3 weeks that BuildBot is not
working for OFBiz at all.
So far I did not run the integration tests on GH because
* we have it on BuildBot
* it's complicated to show the results
Hopefully BuildBot will soon run the OFBiz builds again. But it does not hurt
to run the integration tests on GH, even w/o the results, because at least it's
easier than running them for each commit locally (takes time) and we can run
them locally once we know there is a problem demonstrated with GH actions.
In method filterOutOfStockProducts added case for "virtual" products.
In case a product isVirtual (= Y), then the available inventory count
(lastInventoryCount) is summed up on all its product variants, across
all the ProductFacility records, and then used to determine if the
(parent/virtual) product is in stock or out of stock.
This fix does not check if the facilityId of the ProductFacility
records retrieved is associated and enabled to the ProductStore.
Thanks Nicola Mazzoni for helping in anlyzing this issue.
The previous commit for OFBIZ-12594 was only working on Windows. On *nix OSs
there is no way to reliably get "--test" String from java.class.path property.
Also the previous fix was brittle because relying only on 1 space separating
words.
This fix puts in the SolrDispatchFilter system property at the beginning of the
4 Solr tests and removes it at end of them. That presence can reliably be tested
in ControlFilter that is called before SolrDispatchFilter. It allows to bypass
SecurityUtil::containsFreemarkerInterpolation that would else change the
parameters content type that must be application/x-www-form-urlencoded.
content
Thanks: Tom Pietsch for report and Mart Naum for confirmation
Conflicts handled by hand
If quantity not reserved is not 0, requireInventory is set to Y, system
let an order to be placed anyway.
The usage of the service around the project let make an important
assertion: we can consider that also reserveInventory flag of the
ProductStore is Y, since right now is always the caller context that
checks for the reserveInventory flag.
This fix simply returns a specific error to the caller, avoiding the
system to place the order.
Thanks Nicola Mazzoni for helping in debug this issue and Jacopo
Cappellato for the patch review.
I did not catch an issue put in by this feature because BuildBot is not running
for few days (see INFRA-23076) and i don't always run tests locally (relying on
BuildBot and GH actions don't run tests).
This is the error Solr tests throw else:
<<Unfortunately, the stream was empty / not available. This may be caused by
another servlet filter calling ServletRequest.getParameter*() before
SolrDispatchFilter>>
Conflicts handled by hand
I did not catch an issue put in by this feature because BuildBot is not running
for few days (see INFRA-23076) and i don't always run tests locally (relying on
BuildBot and GH actions don't run tests).
This is the error Solr tests throw else:
<<Unfortunately, the stream was empty / not available. This may be caused by
another servlet filter calling ServletRequest.getParameter*() before
SolrDispatchFilter>>
Conflicts handled by hand
The ProductStore is set up to reserve inventory from more than one
facility, so the flag oneInventoryFacility is set to N.
Flags requireInventory and reserveInventory are both to Y.
Flag allocateInventory is N.
A product can be anyway reserved in a ProductStoreFacility with its
thru date set and past, so is disabled, and if is not present enough
inventory quantity on other facilities.
Added filtering by date on multiple facility code in
reserveStoreInventoryMethod() and on isStoreInventoryAvailable().
This is the first part of the problem resolution; the second part is
accomplished by path to OFBIZ-12455 Jira.
Thanks: Jacques Le Roux and Pierre SMith for Jira feedbacks, Nicola
Mazzoni for helping in bug analaysis and Jacopo Cappellato for path
review.
OFBIZ_12587 is a definitive solution to prevent any kind of Freemarker exploits.
But it's hard to realise because OFBiz exposes objects, like attributes from the
Servlet scopes. So in the meantime preventing Freemarker interpolation in fields
is a pragmatic solution.
This is an improvement but needs to be backported because it kinda affects
security
Conflicts handled by hand
SeoContextFilter.java
ControlFilter.java
When I worked with Mathieu I did not measure how it will be hard sometimes to
backport later :/
Also due to checkstyle module to MODULE change is always a pain in the ass :/
OFBIZ_12587 is a definitive solution to prevent any kind of Freemarker exploits.
But it's hard to realise because OFBiz exposes objects, like attributes from the
Servlet scopes. So in the meantime preventing Freemarker interpolation in fields
is a pragmatic solution.
This is an improvement but needs to be backported because it kinda affects
security
Conflicts handled by hand
SeoContextFilter.java
ControlFilter.java
When I worked with Mathieu I did not measure how it will be hard sometimes to
backport later :/
For the next demo to work we need to set the host-headers-allowed to
demo-next.ofbiz.apache.org instead of demo-old.ofbiz.apache.org
Thanks to Drew Foulks for handling the Infra part for us
Qing Xu, a security reporter, alerted us that, despite no current vulnerability,
it could be maybe possible to do DOS attacks using Java deserialisation.
That has been fixed with https://openjdk.java.net/jeps/290 and even implemented
in Java 8, but it needs a little effort on our side, here it is.
There is no real bug, to this is an improvement but a security improvement so it
will be backported
Thanks: Qing Xu for report
Conflicts handled by hand gradle.properties
Contains bugfixes and improvements, see
https://logging.apache.org/log4j/2.x/changes-report.html#a2.17.2
This introduces the exclusion of some compile
dependencies (bountycastle bc*-jed14) for the
itext library which interfered with newer
versions from another dependency.
Changes name="upload_file_type_bogus" in EditCategory.ftl and
EditProductContent.ftl to name="up_load_file_type_bogus"
The last line of deniedWebShellTokens in SecuredUpload and similarly in
SecurityUtilTest classes should no have a comma and anti-slash as previous lines.
Also logs when an image contains one of the token in deniedWebShellTokens
Removes non satisfying last line (because images may contain those strings) in
security.properties and SecurityUtilTest.java
In SecuredUpload, renames deniedFileExtensions and deniedWebShellTokens to
respectively getDeniedFileExtensions and getDeniedWebShellTokens
Demo content website cmssite/cms could not be accessed because in this case
requestUri is null. Like login request it does not need auth, I did not dig
further, here is a simple fix.
Adds <<",","+",',','+'>> to deniedWebShellTokens as an obviously non satisfying
(because images may contain those strings, I checked) temporary solution before
looking at Freemarker::WhitelistMemberAccessPolicy as suggested by Matei
Thanks to Matei "Mal" Badanoiu for reporting this post-auth vulnerabily
A user with rights to modify and/or create websites may insert malicious HTML
elements in the “webappPath” parameter from content/control/EditWebSite
resulting in XSS.
In order to trigger the XSS a victim needs to navigate to main page of the
modified website (eg webpos or ecommerce) and interact with the malicious
HTML elements (eg trigger the “onmouseover” event by navigating with the mouse
over the “form” and/or “a” tags).
Thanks to Matei "Mal" Badanoiu for reporting this post-auth vulnerabily
Conflicts handled by hand in EntityAutoEngine.java
This was already fixed but we can do better by using SecuredUpload::isValidText
Also grabs some tokens to security.properties::deniedWebShellTokens from
DataEvents.java, and adds "alert" even it's harmless as is, as it's often used
by attackers to demonstrate issues, so will earn time.
Though an improvement, I backport as it's related to security
Conflicts handled by hand in data/DataEvents.java
By inserting malicious content in the “Text” field from
“/content/control/updateLayoutSubContent” -> “Templates”, an attacker may
perform SSTI (Server-Side Template Injection) attacks, which can leverage
FreeMarker exposed objects to bypass restrictions and obtain RCE (Remote Code
Execution).
This fixes it by calling SecuredUpload::isValidText on the “Text” field content.
I'll check that there are no other attack opportunities...
Thanks: Mal Aware <awaremal@gmail.com> for reporting this post-auth vulnerabily
Conflicts handled by hand in ContentManagementServices.java
Adds some tokens in security.properties::deniedWebShellTokens
Updates SecurityUtilTest::webShellTokensTesting accordingly
# Conflicts handled by hand
framework/security/config/security.properties
framework/security/src/test/java/org/apache/ofbiz/security/SecurityUtilTest.java
Trivial change in DataServices.java: uses the SecuredUpload import in 2 places
it was missing.
Adds some tokens in security.properties::deniedWebShellTokens
Removes a duplicated htaccess in security.properties::deniedFileExtensions
Last commits put in an issue in DataServices.java and GroovyBaseScript.groovy.
I should have used SecuredUpload::isValidFileName with dataResourceName and
SecuredUpload::isValidFile with objectInfo. This fixes that.
Also fixes formatting checkstyle issues in SecuredUpload class and move allow
all and check line length in right places (it's about content)
In security.properties, adds some deniedFileExtensions and improves few comments
(easier to merge later)
In DataServices::createFileNoPerm filters file upload to verify the filename
before the content is checked if necessary
Manually merges SecuredUpload.java based on trunk
In previous comment, to prevent linked images inside SVG I used xlink:href
That's no good, it must be followed by <<="http">>, this does it. Else a lot
of SVP images would not be loadable. Because xlink:href can be used in other
cases.
I reopened, forgot that you need to set -PjvmArgs, as I commented before:
<<For a 2GB+ file you need to set Xmx to more than the double of the file size,
eg: gradlew ofbiz -PjvmArgs="-Xms1024M -Xmx5048M">>
This also adds information in security.properties:
<<By default (OOTB) OFBiz is protected against Large File Denial of Service>>
After reviewing https://docs.oracle.com/cd/B14099_19/web.1012/b14014/jspxml.htm,
since we use "<jsp:" this removes "scriptlet>", "declaration>" and "expression>".
They are redundant with "<jsp:"; "<jsp:" covers those already.
Also adds eval( for js and shrinks base64_decode to decode, and processbuilder
to process because of OFBIZ 12571
I always test the denied tokens against a lib of images with 33 600 elements (
3.99 GB). If an image contains the token it can't be in the denied list.
Simpler for now, later OFBIZ 12573 needs to be done
Also formatting improvement in DataResourceWorker.java: removes a blank lines
Thanks: Tim Allison <tallison@apache.org> for the nice exchanges :)
The 1st issue was due to use of processbuilder token. It has been added to
deniedWebShellTokens in security.properties by f2cf262 commit for OFBIZ 11948
The tokens function (for js) and class have been added since while browsing
https://github.com/tennc/webshell
As mention the related deniedWebShellTokens TODO comment: "TODO.... to be continued
with known webshell contents... a complete allow list is impossible anyway...
So, later a deeper review of Groovy sandbox possibilities will be done..
Thanks: Y4er for report