10730 Commits
Author SHA1 Message Date
Deepak Dixit 9b79d38ec5 Updated notification tag maxOccurs, as system allow multiple notification per service 2022-07-29 15:11:52 +05:30
Deepak Dixit 6013c74c22 Improved: Updated mysql jdbc-driver class from com.mysql.jdbc.Driver to com.mysql.cj.jdbc.Driver (OFBIZ-12675) (#529)
MySQL Connector/J 8.0 is highly recommended for use with MySQL Server 8.0 and 5.7. Please upgrade to MySQL Connector/J 8.0.
https://dev.mysql.com/doc/connector-j/8.0/en/
2022-07-29 14:49:59 +05:30
Jacques Le Roux df98889c7b Improved: Data of tenant specific component gets loaded in all instances (OFBIZ-6065)
Improves the information provided about importing and loadAll Gradle task
loadTenant Gradle task should be used when necessary
2022-07-28 09:40:28 +02:00
Jacques Le Roux 9351b641e8 Improved: Data of tenant specific component gets loaded in all instances (OFBIZ-6065)
The tenant relevant Gradle tasks are:

OFBiz Server tasks
------------------
createTenant - Create a new tenant in your environment
loadAdminUserLogin - Create admin user with temporary password equal to ofbiz.
You must provide userLoginId
loadAll - Load default data; meant for OFBiz development, testing, and demo
purposes
loadTenant - Load data using tenantId

loadAll is the new "ant load-data" and should clearly only used for
"OFBiz development, testing, and demo purposes".

Remains the import functionalities. We don't want to get to far with that,
a warning is enough.

Thanks: Pierre Smits for report
2022-07-27 16:20:55 +02:00
georg1312 d82626974a Improved: Strip data to maximum column length for initialRequest and (#526)
initialReferrer (OFBIZ-12672)

Analogous to the existing limitation of the "initialUserAgent" to 250
characters, installation of a limitation of 2000 characters for
initialRequest and initialReferrer, so as not to exit with
"MysqlDataTruncation: Data truncation: Data too long for column
'INITIAL_REFERRER' at row 1" in the case of very long requests.
2022-07-25 09:14:08 +02:00
Michael Brohl 20ccb4dcfd Improved: Make loading of data containing urls configurable
(OFBIZ-12670)

Introduces a SystemProperty security#security.datafile.loadurls.enable
which can be set to true to allow loading of urls in the XML import.
2022-07-21 15:14:58 +02:00
Nicolas Malin e12c98ac5b Implemented: Define return user message from controller (OFBIZ-12652)
Currently, when you wish return a message to a user after an event request, you need to set it ine the called event.

for a service in java :
  ServiceUtil.returnSuccess("Your service is a success")

for a service in groovy :
  return success("Your service is a success")

for a Java class :
  request.setAttribute("_EVENT_MESSAGE_", "Your service is a success");

If during an integration, you want to use standard service like createProduct, createPartyRelationship, and need a specific message for users, you need to define your own service.

For escape this case and increase the service usability, I propose to implement a new system to override the event return by a definition depending on the buisness context.

For that two improvement :
1. Add new child element to request-map->response on the controller
    With given the exact value:

         <response name="success" type="request" value="json">
             <return-user-message value="Your service is a success"/>
         </response>

    With a flexible expander:

        <response name="success" type="request" value="json">
           <return-user-message value="Your service to change is a success"/>
        </response>

    With a property:

        <response name="success" type="request" value="json">
           <return-user-message ressource="CommonUiLabels" value="CommonSuccessfullyCreated"/>
        </response>

    From a context field:

        <response name="success" type="request" value="json">
           <return-user-message from-field="mySpecificReturnMessage"/>
        </response>

2. From the context directly sent from the form

  <form name="CallEvent" target="MyEvent" .. >
     <field name="_CUSTOM_EVENT_MESSAGE_"> <hidden value="Your service to change is a success"/>
     <field name="_CUSTOM_ERROR_MESSAGE_"> <hidden value="Your service failed"/>

Thanks to Florian Motteau for the implementation help
2022-07-15 15:59:09 +02:00
Nicolas Malin 2c7d552840 Fixed: Calendar in Work Effort does not work (OFBIZ-12665)
Fix a null pointer exception when a menu extend a menu with empty item.
In this case, we use a empty link for the extended menu.

Thanks to Jacques Leroux for the alert
2022-07-15 14:38:16 +02:00
Nicolas Malin f1c2e8732e Improved: Allow DynamicView use in standard performFind services (OFBIZ-12663)
Extend standard find services (prepareFind, executeFind and performFind) to use a built dynamicView instead of just an entityName.

With this, you can prepare a dedicated dynamicView and call the performFind with on result a database optimisation.

Thanks to Leila Mekika for this proposal and patch
2022-07-08 17:33:30 +02:00
Nicolas Malin 576307ac5c Improved: Add missing parameter on xsd for widget-form:on-field-event-update-area (OFBIZ-12586)
On the widget-form.xsd definition, the element on-field-event-update-area haven't possibility to define some parameters.

But on java code, the on-field-event-update-area is read like on-event-update-area.

If you put some parameters on your code, your editor raise a syntax error but OFBiz analyse and execute it well.

            <wf:field name="reload" >
                <wf:submit/>
                <wf:on-field-event-update-area event-type="click"
                                               area-target="ReloadIt">
                    <wf:auto-parameters-service service-name="computeForReload" send-if-empty="false"/>
                </wf:on-field-event-update-area>
            </wf:field>

By the way we can synchronize the on-field-event-update-area xsd on on-event-update-area.
2022-07-01 18:31:41 +02:00
Nicolas Malin e0f61dceb7 Improved: Convert a display-entity to hyperlink if subhyperlink this empty (OFBIZ-12547)
When you define a display-entity you have the possibility to add a sub-hyperlink related to the element displayed.

             <field name="partyIdFrom">
                <display-entity entity-name="PartyNameView" key-field-name="partyId" description="${groupName}${firstName} ${lastName}">
                    <sub-hyperlink target="viewprofile" description="view">...</sub-hyperlink>
                </display-entity>
            <field>

This display the party name with a link "view" connected.

When you use the name as link without a dedicate sub link, you need to change all the form to resolve the information to display for creating an hyperlink

            <row-actions>
                <entity-one entity-name="PartyNameView" value-field="partyName">
                        <field-map field-name="partyId" from-field="partyIdFrom"/>
                </entity-one>
                <set field="partyNameValue" value=" ${partyName.groupName}${partyName.firstName} ${partyName.lastName}"/>
            </row-actions>
            <field name="partyIdFrom">
                <hyperlink target="viewprofile" description="${partyNameValue}">..</hyperlink>
            </field>

We lost the advantage of the display-entity. With this case, we improve display-entity with the following rule :

If the display-entity has a subhyperlink and the subhyperlink haven't description, the description of the link is the display-entity description.
In the code, if a sub-hyperlink has an empty description, we convert the display entity as a hyperlink
2022-07-01 18:21:49 +02:00
Nicolas Malin 7ea371bde3 Fixed: MenuItem doesn't follow correctly extended informations (OFBIZ-12628)
Fix a null pointer exception when the extended menu have an entry without link.

Thanks to Jacques Leroux for the alert
2022-07-01 17:37:14 +02:00
Jacques Le Roux 78ee6f8792 Improved: CustomSafePolicy, also use TagBalancingHtmlStreamEventReceiver (OFBIZ-12653)
Adds <img> and <hr> to CustomSafePolicy, removes obsolete <tt>. <img> allows
only attributes src and alt.

Both <br> and <br /> are correct. For that, this rather uses
TagBalancingHtmlStreamEventReceiver

Thanks: Ingo Wolfmayr
2022-07-01 17:26:18 +02:00
Jacques Le Roux 18d0d75f43 Improved: Update build.gradle to the latest dependencies (OFBIZ-12658)
I noticed that JpegImageParser from Apache Commons Imaging never had a write
option. So I commented it out. I tested uploading, it's OK

org.apache.axiom.om.impl.builder.StAXOMBuilder has been replaced by
org.apache.axiom.om.OMXMLParserWrapper
in SOAPEventHandler.java

The rest of information is in the main build.gradle

I got 2 test errors on Win7. I'll see how it goes on GH and BB.
Maybe it's write.lock issues with Lucene or/and Solr indexes
2022-06-27 15:40:08 +02:00
Jacques Le Roux 14c58a11c8 Improved: OWASP sanitizer breaks proper rendering of HTML code (OFBIZ-12653)
Allows both <br> and <br /> to pass in UtilCodec::checkStringForHtmlSafe, both
are correct.

Clarifies owasp.properties documentation about how to create own sanitizer
policies
2022-06-21 20:57:37 +02:00
Nicolas Malin 41dd06b72f Fixed: MenuItem doesn't follow correctly extended informations (OFBIZ-12628)
When you use two menus where the second extend the first, menu items and menu links aren't correctly propagated.

The menu items and menu links presents on the second menu already have the first menu on their model.

    ****
    <menu name="FirstMenu" extends="CommonInlineBarMenu" extends-resource="component://common/widget/CommonMenus.xml">
        <menu-item name="MyItem">
            <link target="GoAction"/>
        </menu-item>
    </menu>
    <menu name="SecondMenu" extends="FirstMenu"/>
    ****

The result, if during the rendering some information are generated from the menu, in the previous case, it's always the "FirstMenu" that would be use.

Like when your menu generate a hidden-form, the SecondMenu link the bad form

    ****
    <form method="post" action="GoAction" onsubmit="javascript:submitFormDisableSubmits(this)" name="SecondMenu">...</form>
    <a href="javascript:ajaxSubmitFormUpdateAreas('FirstMenu', 'xxx')">MyItem</a>
    ****

To solve it and don't break the thread safe pattern, I introduce two new constructor for ModelItem and MenuLink for duplicate the ModelMenuItem and MenuLink in memory with the new parent. With this, the SecondMenu iw now completely duplicated on memory and now share any reference with the extended menu.

Thanks to Marco Rodrigues that detect this problem
2022-06-17 11:19:22 +02:00
Jacques Le Roux 84d30a5abc Improved: Unable to upload a file through ecommerce (OFBIZ-12636)
Allows to check for only CSV files. This is not used OOTB in OFBiz.

Thanks: Sachin for report and confirmation it's OK
2022-06-16 17:43:49 +02:00
Jacques Le Roux 658552e4fb Improved: Secure the uploads (OFBIZ-12080)
Few more debug log texts changes for clarity

I backport to ease future merging while backporting
2022-06-11 14:45:09 +02:00
Jacques Le Roux e3f5d31db1 Fixed: Secure the uploads (OFBIZ-12080)
Several changes in SecuredUpload class:
Mostly checks that uploaded CSV files are valid text files
Also few debug log texts changes for clarity
2022-06-11 11:09:26 +02:00
Jacques Le Roux 61645a72c4 Fixed: Unable to upload a file through ecommerce (OFBIZ-12636)
We need to upload a CSV format file as part of project requirement
We were able upload CSV format file under eCommerce in Version 16.11.07.
We are trying to implement the same in version 18.12.05 - there the same upload
module does not work.

jleroux: I missed to check and allow CSV files when all files are allowed.
That now uses:
https://commons.apache.org/proper/commons-csv/apidocs/org/apache/commons/csv/CSVFormat.html

The default CSV format used to check upload of CSV files is set in the property
csvformat=CSVFormat.DEFAULT in security.properties

Thanks: Sachin for report
2022-06-11 08:57:36 +02:00
Deepak Dixit 5e29578ad6 Fixed: Added missing notification tag entry in services.xsd file (OFBIZ-12635) 2022-06-09 15:28:24 +05:30
Jacques Le Roux ae146e02ef Improved: XML Import fails due to security check (OFBIZ-12602)
Hide the "Received a null Security object from HttpServletRequest" warning when
ControlFilter is in the StackTrace.
2022-06-07 12:04:05 +02:00
Leila 3cc30333d0 Manage groovy DSL error message retrieval (OFBIZ-10692)
Add integration test to validate the good retrieval of the success /
error message.

Ajout de 4 services groovy pour tester le fonctionnement du moteur de service en groovy couplé a des appels de type DSL
2022-05-27 16:27:03 +02:00
Jacques Le Roux 088b22235d Improved: Json response failed when element not serializable (OFBIZ-12621)
No functional changes, just removing a compilation warning

While compiling with Java 11 I got this message:
> Task :compileJava
UtilHttp.java:502: warning: [unchecked] unchecked call to
<R>map(Function<? super T,? extends R>) as a member of the raw type Stream
[...]

This fixes it with the help of Eclipse
2022-05-26 12:24:49 +02:00
Jacques Le Roux b94b6aecaf Fixed: Webtools Service Logs ‘Service Name’ column always empty (OFBIZ-12625)
rs.getModelService().getName() should be used
instead of
rs.getModelService().name

The rest of changes is only non functional formatting and such

Thanks: Emad Radwan for reporting
2022-05-26 11:27:55 +02:00
Nicolas Malin 09cf6eb499 Improved: Redirect unauthenticated user to login on AJAX calls (OFBIZ-12620)
At this time when you aren't logged and you need to be authenticated for access to a page, OFBiz return the login page with a http code 200

This set a problem for ajax call to understand why the page returned isn't the attendee/

This improvement does 3 things :
   * return a HTTP status code 401 (Unauthorized) in LoginWorker::checkLogin if user is not authenticated (no more 200),
   * in common-theme, OfbizUtil.js, on AJAX calls handler, if the request fails with a 401 status code, reload current page (so we land on login form instead of being stuck in a blank modal) with a new search param `clickOn` containing the id of the link that triggered the AJAX call,
   * on page load, if `clickOn` search parameter exists, trigger a click event on the link, so the initial modal pops in after a successful login

Thanks to Florian Motteau for this improvement
2022-05-25 15:38:39 +02:00
Nicolas Malin 82f25f6192 Fixed: Json response failed when element not serializable (OFBIZ-12621)
When you forward a request-map on json response after a service call like this :

     <request-map uri="updateMyEntity">
        <security https="true" auth="true"/>
        <event type="service" invoke="serviceWithUnserializableElement"/>
        <response name="success" type="request" value="json"/>
        <response name="error" type="request" value="json"/>
    </request-map>

The request json works fine if the serviceWithUnserializableElement have an element unserializable on out put at the first level

     <service name="serviceWithUnserializableElement"...>
        <attribute name="outputStream" type="java.io.OutputStream" mode="OUT"/>
     </service>

If the definition is like this

     <service name="serviceWithUnserializableElement"...>
        <attribute name="outputStreamMap" type="Map" mode="OUT"/>
     </service>

The json failed to generate if the outputStream is a map. despite the function UtilHttp.getJSONAttributeMap that safe only the first map level and don't work in deep.

By the way, we improved the getJSONAttributeMap to work in deep by a recursive treatment on Map and List.
2022-05-25 14:35:36 +02:00
Pierre Smits 7973f09eb3 Improved: Have CommonProduct Label for use with LookupProduct function (OFBIZ-12542) (#492)
The LookupProduct function is used across various applications,
intended to assist users with the need to search, find and select
products in forms where one needs to be added or changed.
Also usable in overviews where a product is referenced.

modified: CommonUiLabels.xml
2022-05-22 11:55:42 +02:00
Jacques Le Roux 7a3504eb8a Improved: Required field not working on upload type form (OFBIZ-12619)
Fixes a checkstyle issue
2022-05-21 10:39:58 +02:00
Jacques Le Roux 869a1ea4e4 Fixed: Required field not working on upload type form (OFBIZ-12619)
When trying to submit an upload form with empty required fields, the validation
is not blocked and the submit is made.

Thanks: Leila Mekika
2022-05-21 10:35:11 +02:00
Jacques Le Roux ad019a2217 Improved: Fixes typos 2022-05-05 19:36:03 +02:00
Jacques Le Roux 5408ddb449 Improved: Secure the uploads (OFBIZ-12080)
I did not have these ones, found there
https://github.com/swisskyrepo/PayloadsAllTheThings/blob/master/Methodology%20and%20Resources/Reverse%20Shell%20Cheatsheet.md
and there https://github.com/tennc/webshell
2022-04-25 10:56:09 +02:00
Jacques Le Roux 1514824b7d Fixed: no functional changes, fixes chekstyle issues
Also adds javadoc task to GH actions, let's see if it works there.
2022-04-22 18:33:57 +02:00
Nicolas Malin 6f7f03b742 Fixed: RequestMap.event with type service multi doesn't work with dynamic url (OFBIZ-12604)
When you define a controller request-map with an event of type 'service-multi', if your uri contains multiple allocation like 'MyWay/MyAction' the class ServiceMultiEventHandler failed to execute with an EventHandlerException

 *****
    <request-map uri="Payment/QuickSend">
        ...
        <event type="service-multi" invoke="quickSendPayment"/>
        ...
 *****

The reason comes from the necessary to resolve the attribute global-transaction on event definition, and to do that a call to ConfigXMLReader is realized.

But unnecessary because we already have the event element on the context.
Use it directly, simplify the code and fix this issue
2022-04-22 16:09:53 +02:00
Jacques Le Roux bbe05496fb Improved: no functional change, fixes a Javadoc issue
SecurityUtil.java:91: error: unknown tag: String
     * @return List<String>
2022-04-22 12:31:53 +02:00
Jacques Le Roux 3de52be907 Fixed: XML Import fails due to security check (OFBIZ-12602)
When importing an entity with "${" in for at least an element it's rejected
because of the security check done to protect from Freemarker unauth attacks
(see OFBIZ-12594).

As suggested by Ingo, allowing users with appropriate permissions seems an
usable solution. We still need to define the "appropriate permissions".
We can start with OFBTOOLS and WEBTOOLS, as it's reported by Ingo, and add
others later if they ever come.

Thanks: Ingo Wolfmayr for report and suggestion
2022-04-20 13:58:48 +02:00
Jacopo Cappellato 1a567ed42e Improved: In UtilHttp, for regex processing of urls, replace Java regex with Google RE2J. (#512)
The Main advantage of RE2J is that it guarantees linear time execution.
2022-04-14 07:29:09 +02:00
Jacques Le Roux da45698dc5 Fixed: Test run was unsuccessful because of failing solr tests (OFBIZ-12595)
The previous commit for OFBIZ-12594 was only working on Windows. On *nix OSs
there is no way to reliably get "--test" String from java.class.path property.

Also the previous fix was brittle because relying only on 1 space separating
words.

This fix puts in the SolrDispatchFilter system property at the beginning of the
4 Solr tests and removes it at end of them. That presence can reliably be tested
in ControlFilter that is called before SolrDispatchFilter. It allows to bypass
SecurityUtil::containsFreemarkerInterpolation that would else change the
parameters content type that must be application/x-www-form-urlencoded.
content

Thanks: Tom Pietsch for report and Mart Naum for confirmation
2022-04-13 14:38:19 +02:00
Jacques Le Roux b5370cbf51 Fixed: Prevent Freemarker interpolation in fields (OFBIZ-12594)
I did not catch an issue put in by this feature because BuildBot is not running
for few days (see INFRA-23076) and i don't always run tests locally (relying on
BuildBot and GH actions don't run tests).

This is the error Solr tests throw else:
<<Unfortunately, the stream was empty / not available. This may be caused by
another servlet filter calling ServletRequest.getParameter*() before
SolrDispatchFilter>>

Got an issue with previous commit (I guess confusion with 22.01)
2022-04-06 13:36:21 +02:00
Jacques Le Roux e5b0d63b65 Fixed: Prevent Freemarker interpolation in fields (OFBIZ-12594)
I did not catch an issue put in by this feature because BuildBot is not running
for few days (see INFRA-23076) and i don't always run tests locally (relying on
BuildBot and GH actions don't run tests).

This is the error Solr tests throw else:
<<Unfortunately, the stream was empty / not available. This may be caused by
another servlet filter calling ServletRequest.getParameter*() before
SolrDispatchFilter>>
2022-04-06 10:59:56 +02:00
Jacques Le Roux a3068004a1 Fixed: testInvoicePerShipmentSetFalse work with runtime configuration data (OFBIZ-12453)
UtilProperties::setPropertyValueInMemory was only based on files. Thinking by
analogy, as Pierre pushed SystemProperties with OFBIZ-6189, I thought that by
adding an UtilProperties::setPropertyValueInMemory based on
EntityUtilProperties::getProperties it would resolve the issue. But it did not.

Looking closer at the code, I wondered why UtilProperties::setPropertyValueInMemory
was used at all.

This fixes the issue by simply removing setPropertyValueInMemory and setting "Y"
or "N" when calling testInvoicePerShipment.

It though keeps the new UtilProperties::setPropertyValueInMemory variant
2022-04-06 09:31:30 +02:00
Jacques Le Roux 2aeb282cdc Improved: Prevent Freemarker interpolation in fields (OFBIZ-12594)
OFBIZ-12587 is a definitive solution to prevent any kind of Freemarker exploits.
But it's hard to realise because OFBiz exposes objects, like attributes from the
Servlet scopes. So in the meantime preventing Freemarker interpolation in fields
is a pragmatic solution.

This is an improvement but needs to be backported because it kinda affects
security
2022-04-04 08:04:39 +02:00
Jacques Le Roux 5d4dcd2ed4 Improved: just a tiny comment change in security.properties
Make clear that it's impossible to create a complete deniedWebShellTokens
2022-04-02 16:24:17 +02:00
Jacques Le Roux 455bfbe8b8 Fixed: NullPointerException in plugin 'projectMgr' when listing TimeSheets (OFBIZ-12593)
The empty string causes a NullPointerException, because in the WidgetWorker
returns only an anchorElement  if the description of the hyperlink is not empty.

Solution:  If the anchorElement is null, the line is simply not executed:

Thanks: Tom Pietsch
2022-04-02 11:37:32 +02:00
kabutz 0e6732b526 Removed direct call to Class.newInstance() (#510) 2022-03-31 19:16:17 +02:00
Jacques Le Roux 724ea1ffea Implemented: Restart the ofbiz-vm3 demos server (OFBIZ-22722)
For the next demo to work we need to set the host-headers-allowed to
demo-next.ofbiz.apache.org instead of demo-old.ofbiz.apache.org

Thanks to Drew Foulks for handling the Infra part for us
2022-03-28 18:47:17 +02:00
Jacques Le Roux f49bb67662 Fixed: Secure the uploads (OFBIZ-12080)
Changes name="upload_file_type_bogus" in EditCategory.ftl and
EditProductContent.ftl to name="up_load_file_type_bogus"

The last line of deniedWebShellTokens in SecuredUpload and similarly in
SecurityUtilTest classes should no have a comma and anti-slash as previous lines.

Also logs when an image contains one of the token in deniedWebShellTokens
2022-03-17 18:09:35 +01:00
Jacques Le Roux 65ae69185c Fixed: Stored XSS in webappPath parameter from content/control/EditWebSite (OFBIZ-12584)
Adds "webappPath" token in deniedWebShellTokens to definitely fix this issue

Also adds "assign" token  for possible future Freemarker exploits
2022-03-13 21:10:21 +01:00
Jacques Le Roux 33fe2aaba5 Fixed: Remove _PREVIOUS_REQUEST_ Session Attribute on non-authentication pages (OFBIZ-12047)
Demo content website cmssite/cms could not be accessed because in this case
requestUri is null. Like login request it does not need auth, I did not dig
further, here is a simple fix.
2022-03-06 09:30:30 +01:00
Jacques Le Roux 0380ab85fd Fixed: Secure the uploads (OFBIZ-12080)
Removes non satisfying last line (because images may contain those strings) in
security.properties and SecurityUtilTest.java

In SecuredUpload, renames deniedFileExtensions and deniedWebShellTokens to
respectively getDeniedFileExtensions and getDeniedWebShellTokens
2022-03-03 17:22:50 +01:00