Skip to Main Content
IBM System Storage Ideas Portal


This portal is to open public enhancement requests against IBM System Storage products. To view all of your ideas submitted to IBM, create and manage groups of Ideas, or create an idea explicitly set to be either visible by all (public) or visible only to you and IBM (private), use the IBM Unified Ideas Portal (https://ideas.ibm.com).


Shape the future of IBM!

We invite you to shape the future of IBM, including product roadmaps, by submitting ideas that matter to you the most. Here's how it works:

Search existing ideas

Start by searching and reviewing ideas and requests to enhance a product or service. Take a look at ideas others have posted, and add a comment, vote, or subscribe to updates on them if they matter to you. If you can't find what you are looking for,

Post your ideas
  1. Post an idea.

  2. Get feedback from the IBM team and other customers to refine your idea.

  3. Follow the idea through the IBM Ideas process.


Specific links you will want to bookmark for future use

Welcome to the IBM Ideas Portal (https://www.ibm.com/ideas) - Use this site to find out additional information and details about the IBM Ideas process and statuses.

IBM Unified Ideas Portal (https://ideas.ibm.com) - Use this site to view all of your ideas, create new ideas for any IBM product, or search for ideas across all of IBM.

ideasibm@us.ibm.com - Use this email to suggest enhancements to the Ideas process or request help from IBM for submitting your Ideas.

Status Not under consideration
Created by Guest
Created on Jul 21, 2017

Make filename handling independent of locale settings

I previously raised this as a bug in TSM with PMR 60127,040,678 but was instructed to raise it with RFE community instead, so here it goes.

The spectrum protect client does not back up all valid files when the locale settings are set to a multibyte locale (such as UTF-8). On a machine with filenames in mixed character sets, backups of any file with a name that is not a valid UTF-8 string fail if the locale of the dsmc process is UTF-8.

This is documented in the BA manual here:
https://www.ibm.com/support/knowledgecenter/en/SSGSG7_7.1.1/com.ibm.itsm.client.doc/c_cfg_nlsupportunix.html

This behaviour is wrong. Filenames are not strings, but byte sequences. Any byte sequence is a valid filename, regardless of system locale. stat() and open() behave the same regardless of the LANG setting.

The problem can be recreated easily on any machine with something like the following (using bash/sh shell and python):
---
cd `mktemp -d`
export LANG='en_US.UTF-8'
cat <<EOF >create_iso88591.py
#!/usr/bin/env python2
name = '\xC6'
with open(name, 'w') as f:
f.write('test')
EOF
python2 create_iso88591.py
ls -l
sudo dsmc incr '*'
---

If the above snippet is run, the created file will not be successfully backed up by dsmc. An error will be printed to the terminal.

Most new machine deployments have UTF-8 locales by default, yet it is very common to have files in mixed character sets due to legacy issues, and even multiple programs with different locale settings running on the same machine. In the current situation, IBM-provided init scripts need to be modified locally by the system administrator to run in a single-byte locale to successfully back up such systems. This is a heavy maintenance burden with a lot of nodes, especially since these changes need to be verified and possibly re-applied on every client update. Furthermore, it means that it is impossible to get terminal and log output in UTF-8 for these machines, which would be desireable.

From my local testing, it does not appear that the BA client does any transcoding of filenames when restoring from a different locale, so there is no obvious reason why filenames would be parsed as strings in this case.

Please change the filename handling in BA client so it can back up any file on the filesystem regardless of filename and locale settings.

Idea priority High
  • Guest
    Reply
    |
    Jul 27, 2017

    IBM has evaluated the request, and has determined that it can not be implemented at this time or does not align with the current multi-year strategy. This request may be resubmitted for consideration after 18 months from the date of submission. The request submitter will automatically be notified by email when the request qualifies for reconsideration and resubmission.